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Appendix A. Additional Tables 


Appendix A. Additional Tables 


A-1 


Table A.3.1. Number and percentage distribution of data collection paradata, by 
address type: 2019 
Address type 


Observed, occupied, 
Addresses that participated _ residential, nonrespondent 
in an interview addresses! 


Number of Percentageof Numberof Percentage of 


Data collection paradata addresses addresses _ addresses addresses 

Total 85 100.0 440 100.0 
Nonresponse study site 

Texas interview site 25 27.1 70 15.3 

Connecticut 20 23.5 70 14.8 

California 15 16.5 60 14.1 

Ohio 30 32.9 70 15.7 

Texas observation site tT tT 60 13.4 

Illinois t t 60 12.5 

Washington, DC t t 60 14.1 
Nonresponse study interview language 

English only 70 83.5 t t 

Spanish only 10 10.6 iF F 

Mix of English and Spanish 5 5.9 t T 
NHES:2019 final screener response status 

Responded 70 17.7 # # 

Did not respond 15 82.4 440 100.0 
NHES:2019 undeliverable as addressed (UAA) 

mailings status 

One or more UAA mailings 5 3.5 # # 

No UAA mailings 80 96.5 440 100.0 
NHES:2019 screener mailings language 

All bilingual mailings (English and Spanish) 25 29.4 170 38.3 

Mix of bilingual mailings and English-only mailings 45 50.6 220 49.4 

All English-only mailings 15 20.0 50 12.3 

tNot applicable. 


#Rounds to zero. 

1This analysis is restricted to addresses for which household attributes were collected; household attributes were not collected for addresses that could not 
be observed (e.g., cannot locate the address) or for which only a partial address observation could be completed (e.g., can observe the multi-unit building 
but cannot gain entry to observe the sampled unit). Household attributes also were not collected for addresses that were observed to be nonresidential or 
temporarily or permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed 
(UAA) NHES:2019 mailing were excluded from this analysis. 

NOTE: Sample sizes are rounded to the nearest 5 (interviewed addresses) or 10 (observed addresses). Percentages are rounded to one decimal place but 
have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


Table A.3.2. Number and percentage distribution of household-level auxiliary data, 


by address type: 2019 


Household-level auxiliary data2 
Total 

Age of head of household 
18-34 
35-44 
45-54 
55-64 
65 and older 
Missing 

Gender of head of household 
Male 
Female 
Missing 

Education of head of household 
Less than high school credential 
High school credential 
Some college 
Bachelor's degree 
Graduate degree 
Missing 

Race/ethnicity of head of household 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other race, non-Hispanic 
Missing 

Household income 
Less than $50,000 
$50,000-$74,999 
$75,000-$99,999 
$100,000 or higher 
Missing 

Household flagged as having children 
Yes 
No 


Address type 


Addresses that participated in an 
interview 

Number of Percentage of 
addresses addresses 
85 100.0 
10 9.4 
15 20.0 
10 12.9 
15 18.8 
10 11.8 
25 27.1 
25 31.8 
25 30.6 
30 37.6 
30 32.9 
20 21.2 
20 21.2 
10 10.6 
5 5.9 

5 8.2 
35 38.8 
15 18.8 
20 25.9 
5 5.9 
10 10.6 
40 49.4 
10 14.1 
10 12.9 
15 17.6 
5 5.9 
25 30.6 
60 69.4 


Observed, occupied, residential, 
nonrespondent addresses! 


Number of Percentage of 
addresses addresses 
440 100.0 
40 9.8 
70 15.3 
80 18.7 
60 14.4 
50 11.9 
130 30.1 
140 32.4 
140 32.8 
150 34.9 
110 24.4 
70 15.7 
100 23.7 
60 12.8 
40 8.0 
70 15.5 
150 34.2 
80 17.8 
110 25.3 
30 6.6 
70 16.2 
180 40.8 
60 13.7 
50 11.4 
110 26.0 
40 8.2 
130 29.2 
310 70.8 


See notes at end of table. 


Table A.3.2. Number and percentage distribution of household-level auxiliary data, 
by address type: 2019—Continued 


Address type 


Addresses that participated inan Observed, occupied, residential, 


interview nonrespondent addresses? 
Number of Percentage of Number of Percentage of 

Household-level auxiliary data? addresses addresses addresses addresses 
Number of adults in household 

1 adult 55 62.4 240 54.9 

2 adults 15 20.0 110 25.5 

3 or more adults 10 11.8 50 11.4 

Missing 5 5.9 40 8.2 
Phone number available 

Yes 65 77.6 290 66.7 

No 20 22.4 150 33.3 
Route type 

Street 55 65.9 330 74.3 

High rise 30 34.1 110 25.7 
Dwelling type 

Single-unit 55 63.5 310 70.2 

Multi-unit 30 36.5 130 29.8 
Home tenure 

Own 35 43.5 260 58.5 

Rent 40 49.4 140 30.8 

Missing 5 7.1 50 10.7 


1This analysis is restricted to addresses for which household attributes were collected; household attributes were not collected for addresses that could not 
be observed (e.g., cannot locate the address) or for which only a partial address observation could be completed (e.g., can observe the multi-unit building 
but cannot gain entry to observe the sampled unit). Household attributes also were not collected for addresses that were observed to be nonresidential or 
temporarily or permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed 
(UAA) NHES:2019 mailing were excluded from this analysis. 

2The household-level auxiliary data are based on variables available on the NHES:2019 sampling frame. 

NOTE: Sample sizes are rounded to the nearest 5 (interviewed addresses) or 10 (observed addresses). Percentages are rounded to one decimal place but 
have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


A-4 


Table A.3.3. Number and percentage distribution of area-level auxiliary data, by 


address type: 2019 


Area-level auxiliary data 
Total 
Urbanicity2 
Urban 
Suburban 
Town 
Rural 
Region2 
Northeast 
South 
Midwest 
West 
Race/ethnicity stratum’ 
25% or more Black 
40% or more Hispanic 
Other 
Tract poverty rate3 
Less than 20% 
20% or more 
Percent of households in Census block that 
include a child? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Percent of persons in Census block that speak a 
language other than English? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Percent of persons in Census block without a 
high school diploma or the equivalent? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 


Address type 


Addresses that participated 
in an interview 


Number of Percentage of 


addresses 
85 


40 
45 


addresses 
100.0 


47.1 
51.8 


Observed, occupied, 
residential, nonrespondent 


addresses! 
Number of Percentage of 
addresses addresses 
440 100.0 
180 40.1 
240 54.4 
10 1.1 
20 4.3 
70 14.8 
190 42.8 
120 28.3 
60 14.1 
110 25.7 
130 29.4 
200 44.9 
300 68.6 
140 31.4 
100 22.6 
100 22.3 
100 23.5 
140 31.7 
40 9.3 
70 15.7 
140 31.2 
190 43.7 
100 21.6 
90 20.3 
90 21.4 
160 36.7 


See notes at end of table. 


Table A.3.3. Number and percentage distribution of area-level auxiliary data, by address 
type: 2019—Continued 


Address type 
Observed, occupied, 
Addresses that participated residential, nonrespondent 
in an interview addresses! 
Number of Percentage of Number of Percentage of 
Area-level auxiliary data addresses addresses addresses addresses 
Low Response Score* 
First quartile 15 15.3 100 23.1 
Second quartile 10 12.9 80 18.0 
Third quartile 25 28.2 90 20.8 
Fourth quartile 35 43.5 170 38.1 
Residential high-speed internet per 1000 
households> 
600 or less 30 35.3 100 22.8 
601-800 30 37.7 160 35.3 


801 or more 25 27.1 180 41.9 


#Rounds to zero. 

Reporting standards not met. There are too few cases for a reliable estimate. 

1This analysis is restricted to addresses for which household attributes were collected; household attributes were not collected for addresses that could not 
be observed (e.g., cannot locate the address) or for which only a partial address observation could be completed (e.g., can observe the multi-unit building 
but cannot gain entry to observe the sampled unit). Household attributes also were not collected for addresses that were observed to be nonresidential or 
temporarily or permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed 
(UAA) NHES:2019 mailing were excluded from this analysis. 

2These characteristics are based on variables available on the NHES: 2019 sampling frame. 

3These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest 
prevalence of the characteristic in question and those that are in the fourth quartile are those with the highest prevalence. 

4The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. A higher score corresponds to a lower expected mail return rate. This variable was not available for a very small number of cases, and these cases 
have been excluded from this analysis. 

5This characteristic is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: Sample sizes are rounded to the nearest 5 (interviewed addresses) or 10 (observed addresses). Percentages are rounded to one decimal place but 
have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 


A-6 


Table A.3.4. Number and percentage distribution of structure type observation for 


addresses that participated in an interview: 2019 


Structure type Number of addresses 


Total 80 
Single-unit 50 
Attached! 10 
Apartment? 25 
Could not determine # 


#Rounds to zero. 
1Attached structures include duplexes, townhouses, and rowhouses. 
2Apartment structures include low-, medium-, and high-rise apartments. 


Percentage of addresses 


100.0 
59.3 
9.9 
30.9 
# 


NOTE: Structure type observations were not collected for addresses that could not be observed (e.g., cannot locate the address). Addresses that were 
observed to be nonresidential were excluded from this analysis. Sample sizes are rounded to the nearest 5. Percentages are rounded to one decimal place 


but have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


A-7 


Table A.3.5. Percentage of addresses that participated in an interview with 
observed household attributes: 2019 


Observed household characteristics, attributes, and interests Percentage of addresses 


Household member characteristics 


Presence of children! 27.4 
Speaking a language other than English2 5.5 
Household member attributes and interests 

Privacy or security concerns3 27.4 
Outdoor living* 274 
Patriotism> 8.2 
Welcoming decor® 2.7 
Internet or television connectivity” 8.2 
Community involvement® 2.7 
Importance of religion? $ 
Pride in education!9 t 
Other outdoor decor! 20.6 


+Reporting standards not met. There are too few cases for a reliable estimate. 
1Addresses where children are thought to be present are those where observers identified indicators that suggest the children live in the sampled unit. 
Examples include toys, bikes, car seats, strollers, outdoor swings/play sets, and child finder stickers for firefighters. 
2Addresses that speak a language other than English are those where observers identified indicators that people in the sampled unit may speak a language 
other than English. Examples include flags for non-English-speaking countries, in-home business signs, yard/window signs, or bumper stickers written in 
another language at the sampled unit; or a large number of flags, signs, or bumper stickers in (or including) another language in the surrounding 
neighborhood (even if indicators are not observed at the sampled unit). 
3Addresses with privacy or security concerns are those where observers identified indicators that privacy or security is important to those living in the 
sampled unit. Examples include surveillance cameras, driveway gates, security company signs or stickers, and "No trespassing" signs. For multi-unit 
buildings, indicators that focused on the larger building in which the unit was located, such as entry buzzers or fences, were not included because they 
could be not directly associated with the sampled unit. 
4Addresses that value outdoor living are those where observers identified indicators that people in the sampled unit spend time outdoors. Examples 
include patio furniture, porch swings/benches, swing sets, sporting goods, and grills. This variable was added during data processing based on patterns 
observed in write-in responses. 

dresses that value patriotism are those where observers identified indicators of American national or state-specific patriotism. They also include those 
with indicators of current or past involvement with the U.S military or pride in the U.S. military. 

dresses with welcoming decor are those where observers noted the presence of an object that welcomes visitors to the sampled unit, such as a welcome 
mat or welcome sign that explicitly says “welcome.” This variable was added during data processing based on patterns observed in write-in responses. 
7Addresses with Internet or television connectivity are those where observers noted the presence of a satellite, cable, DirectTV dish or other electronic 
equipment, or a sign that indicates television or internet connectivity at the sampled unit. This variable was added during data processing based on 
patterns observed in write-in responses. 
8Addresses with community involvement are those where observers identified indicators that people in the sampled unit are involved in the community. 
Examples include political candidate signs or indicators of being involved with charities, kids' sports/clubs, or neighborhood associations. 
°Addresses that value religion are those where observers identified indicators that suggest that religion is important to the people in the sampled unit. 
Examples include “Bless this House” signs, religious figures or statues, or Mezuzahs. This variable was added during data processing based on patterns 
observed in write-in responses. 
10Addresses with pride in education are those where observers identified indicators that education is important to the sampled unit. Examples include 
school, college and/or university flags or stickers; honor roll bumper stickers; or other indicators of pride in a child’s school. 
11Addresses with other outdoor decor are those where observers noted the presence of items used to decorate the exterior of the sampled unit. Examples 
include gnomes, flamingos, yard art, decorative flags, wreaths, Easter decorations, water features, or windchimes. This attribute focuses only on decorative 
items and does not include items that represent another household attribute shown in the table (e.g., American flags were coded under patriotism only). 
This variable was added during data processing based on patterns observed in write-in responses. 
NOTE: Percentages represent the percentage of addresses for which the attribute was observed. Household attributes were not collected for addresses that 
could not be observed (e.g., cannot locate the address) or for which only a partial address observation could be completed (e.g., can observe the multi-unit 
building but cannot gain entry to observe the sampled unit). Household attributes also were not collected for addresses that were observed to be 
nonresidential or temporarily or permanently vacant. The rounded eligible sample size for addresses that participated in an interview is 75. Details may 
not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table A.4.1. Number and percentage distribution of mail access type observation 
for observed, residential, nonrespondent addresses: 2019 


Mail access type Number of addresses Percentage of addresses 

Total 530 100.0 
Mail slot 30 6.0 
Mailbox attached to the home 160 30.9 
Mailbox at the end of the driveway 110 20.0 
Mailbox across the street or at the end of the road 60 10.7 
Mailbox, slot, or room in a multi-unit building 80 14.9 
More than one type of mail access! $ t 
No mailbox or slot in view 40 7.5 
Could not determine 50 9.6 


Reporting standards not met. There are too few cases for a reliable estimate. 

1This category includes addresses where mail could have been received in more than way (e.g., a house that has both a mail slot in the door and a mailbox 
at the end of the driveway). 

NOTE: Mail access type observations were not collected for addresses that could not be observed (e.g., cannot locate the address) or were observed to be 
nonresidential or permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed 
(UAA) NHES:2019 mailing were excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but 
have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


Table A.4.2. Number and percentage distribution of mail access type observation 
for addresses that participated in an interview: 2019 


Mailaccesstype CC CCCCCCC“‘NSCC#CNNmbeer' ofaddrresses____ Percentage of addresses 
Total 80 100.0 
Mail slot or mailbox attached to the home! 40 46.9 
Mailbox at the end of the driveway, across the street, or at the 
end of the road 25 28.4 
Mailbox, slot or room in multi-unit building 5 8.6 
No mailbox or slot in view 10 12.4 
Could not determine 5 3.7 


1A few cases where mail was received in more than one way were categorized under "mail slot or mailbox attached to the home" since all the ways mail 
was received fit that category. 

NOTE: Mail access type observations were not collected for addresses that could not be observed (e.g., cannot locate the address) or were observed to be 
nonresidential or permanently vacant. Sample sizes are rounded to the nearest 5. Percentages are rounded to one decimal place but have not been changed 
to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table A.4.3. Percentage distribution of interview participants’ self-reported 
engagement with example mailings, by example mailing: 2019 


Example mailing 

U.S. Department of Commerce envelope 
Personal notecard envelope 

National Center for Public Opinion Research envelope 
National Council for Education envelope 
Dentist card 

Utility company envelope 

National Customer Support Center envelope 
Insurance envelope 

Candidate forum flyer 

Box store circular 

Mail order catalog 

Quality Bankers envelope 


#Rounds to zero. 


Reporting standards not met. There are too few cases for a reliable estimate. 


Self-reported engagement! 


Would 
engage 
91.8 
89.4 
88.2 
78.8 
71.8 
61.2 
56.5 
55.3 
42.4 
42.4 
41.2 
30.6 


Would 
not engage 
8.2 

PA 

7.1 
20.0 
24.7 
36.5 
42.4 
44.7 
51.8 
52.9 
52.9 
68.2 


Unclear 
# 

3.5 

4.7 

+ 

3.5 

24 


1Engagement was determined based on participants’ reaction to each example mailing. In the small number of cases where more than one household 
member participated in the interview, engagement was based on the primary interview participant's reaction to the mailing. Participants who "would 
engage" are those who stated they would open, read, or keep the mailing. Participants who "would not engage" are those who stated they would not open, 
read, or keep the mailing. Participants who provided conflicting responses (e.g., would open but throw away) were placed in the "would engage" group. 


"Unclear" participants are those who did not clearly state whether they would open, read, or keep the mailing. 


NOTE: The total number of interview participants was 85. Details may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table A.4.4. Percentage of interview participants that would engage with the mailing, by example mailing and selected characteristics: 2019 


Selected characteristics 
Nonresponse study site 
West South Central interview site 
New England interview site 
Pacific interview site 
East North Central interview site 
Nonresponse study interview language 
English only 
Spanish only 
Mix of English and Spanish 
NHES:2019 final screener response status 
Responded 
Did not respond 
Age1 
18-24 
25-34 
35-44 
45-54 
55-64 
65 and older 
Gender! 
Male 
Female 
Refused 
Race/ethnicity1 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other race, non-Hispanic 
Refused 


Number of 
participants 


25 
20 
15 
30 


US. 
Department 
of Commerce 
envelope 


91.3 
90.0 
92.9 
92.9 


93.0 
88.9 


80.0 


93.3 
91.4 


88.9 
86.7 
83.3 
100.0 
100.0 
87.5 


94.1 
90.0 


Personal 
notecard 
envelope 


87.0 
90.0 
100.0 
85.7 


88.7 
88.9 
100.0 


93.3 
88.6 


100.0 
80.0 
94.4 
89.5 
78.6 

100.0 


94.1 
86.0 


National Center 
for Public 
Opinion 
Research 
envelope 


78.3 
95.0 
92.9 
89.3 


90.1 
778 
80.0 


100.0 
85.7 


778 
86.7 
88.9 
94.7 
78.6 
100.0 


91.2 
88.0 


National 
Council for 
Education 
envelope 


78.3 
80.0 
85.7 
75.0 


77.5 
778 
100.0 


80.0 
78.6 


88.9 
80.0 
778 
68.4 
85.7 
75.0 


76.5 
80.0 


Dentist 
card 


60.9 
80.0 
78.6 
714 


70.4 
77.8 
80.0 


66.7 
72.9 


88.9 
53.3 
83.3 
68.4 
714 
62.5 


76.5 
70.0 


Example mailing 


National 

Utility Customer 
company Support Center 
envelope envelope 
52.2 65.2 
85.0 55.0 
85.7 78.6 
39.3 39.3 
59.2 54.9 
66.7 44.4 
80.0 100.0 
66.7 53.3 
60.0 57.1 
88.9 100.0 
60.0 73.3 
72.2 50.0 
52.6 52.6 
50.0 42.9 
37.5 37.5 
64.7 58.8 
60.0 56.0 
+ + 
64.0 56.0 
51.9 48.1 
78.3 69.6 
50.0 66.7 
t t 


Insurance 
envelope 


60.9 
60.0 
64.3 
42.9 


54.9 
55.6 
60.0 


60.0 
54.3 


88.9 
46.7 
61.1 
47.4 
42.9 
62.5 


61.8 
52.0 


Candidate 
forum flyer 


39.1 
60.0 
57.1 
25.0 


40.8 
44.4 
60.0 


40.0 
42.9 


22.2 
46.7 
61.1 
42.1 
21.4 
37.5 


44.1 
42.0 


Box store 
circular 


43.5 
45.0 
50.0 
35.7 


40.8 
55.6 
40.0 


46.7 
41.4 


44.4 
40.0 
55.6 
36.8 
42.9 
25.0 


35.3 
46.0 


Mail order 
catalog 


52.2 
40.0 
50.0 
28.6 


39.4 
55.6 
40.0 


53.3 
38.6 


33.3 
33.3 
55.6 
36.8 
42.9 
37.5 


26.5 
50.0 


Quality 


Bankers 
envelope 


34.8 
30.0 
28.6 
28.6 


31.0 
33.3 


See notes at end of table. 
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Table A.4.4. Percentage of interview participants that would engage with the mailing, by example mailing and selected characteristics: 2019—Continued 


Example mailing 
National Center 


US. for Public National National 
Department Personal Opinion Council for Utility Customer Quality 
Number of — of Commerce notecard Research Education Dentist company Support Center Insurance Candidate Box store Mail order Bankers 
Selected characteristics participants envelope envelope envelope envelope card envelope envelope envelope forum flyer circular catalog envelope 
Education1 
High school or less 40 89.5 92.1 89.5 78.9 84.2 65.8 55.3 68.4 50.0 57.9 50.0 474 
Some college, but no bachelor's degree 25 96.0 80.0 80.0 68.0 52.0 52.0 48.0 44.0 28.0 40.0 52.0 20.0 
Bachelor's degree 15 93.3 93.3 93.3 86.7 73.3 60.0 60.0 40.0 46.7 13.3 13.3 13.3 
Graduate degree 5 85.7 100.0 100.0 100.0 714 714 85.7 57.1 42.9 28.6 + $ 
Enrollment status1 
Enrolled 10 90.0 100.0 90.0 90.0 90.0 80.0 90.0 80.0 50.0 30.0 50.0 40.0 
Not enrolled 75 91.9 87.8 87.8 78.4 70.3 58.1 52.7 51.4 41.9 44.6 40.5 29.7 
Refused $ + $ $ $ + + $ $ $ $ $ + 
Employment status! 
Employed for pay 60 93.1 89.7 87.9 81.0 724 56.9 55.2 53.4 39.7 37.9 36.2 29.3 
Not employed for pay 25 88.9 88.9 88.9 74.1 70.4 70.4 59.3 59.3 48.1 51.9 51.9 33.3 
Household income1 
$30,000 or less 25 91.3 91.3 91.3 78.3 69.6 69.6 65.2 56.5 30.4 56.5 43.5 30.4 
$30,001-$60,000 20 95.5 100.0 90.9 95.5 77.3 63.6 81.8 63.6 54.5 45.5 54.5 31.8 
$60,001-$100,000 15 92.9 85.7 85.7 71.4 64.3 57.1 35.7 35.7 42.9 42.9 28.6 28.6 
$100,001 or higher 15 92.9 92.9 92.9 57.1 714 50.0 57.1 42.9 28.6 21.4 21.4 28.6 
Refused 10 83.3 66.7 75.0 83.3 75.0 58.3 16.7 75.0 58.3 33.3 50.0 33.3 
Language spoken most often by adults in the 
household1 
English 65 92.5 88.1 89.6 76.1 68.7 58.2 53.7 53.7 40.3 43.3 41.8 31.3 
Spanish 15 84.6 92.3 76.9 84.6 84.6 69.2 61.5 61.5 46.2 46.2 46.2 30.8 
English/Spanish equally £ + $ $ $ $ $ $ $ $ $ $ £ 
Other $ + $ $ $ $ $ + $ $ $ $ $ 
Home internet access! 
No access + # + $ $ + + $ $ $ $ $ + 
Phone/tablet access only 20 94.4 94.4 72.2 83.3 77.8 55.6 61.1 55.6 38.9 66.7 50.0 33.3 
Computer 65 90.6 87.5 92.2 78.1 70.3 60.9 54.7 56.3 42.2 34.4 39.1 28.1 
Refused $ # $ $ $ $ $ $ $ $ $ $ $ 


See notes at end of table. 
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Table A.4.4. Percentage of interview participants that would engage with the mailing, by example mailing and selected characteristics: 2019—Continued 


National Center 


Example mailing 


US. for Public National National 

Department Personal Opinion Council for Utility Customer Quality 

Number of of Commerce notecard Research Education Dentist company Support Center Insurance Candidate Box store Mail order Bankers 

Selected characteristics participants envelope envelope envelope envelope card envelope envelope envelope forum flyer circular catalog envelope 

Child in household1 

Yes 45 90.7 88.4 86.0 76.7 69.8 53.5 53.5 55.8 44.2 39.5 37.2 32.6 

No 30 92.9 90.5 90.5 81.0 73.8 69.0 59.5 54.8 40.5 45.2 45.2 28.6 
Number of adults in household! 

1 adult 25 84.0 92.0 84.0 64.0 64.0 60.0 60.0 56.0 44.0 48.0 40.0 20.0 

2 adults 40 95.1 87.8 87.8 85.4 73.2 53.7 48.8 46.3 43.9 36.6 39.0 29.3 

3 or more adults 20 94.7 89.5 94.7 84.2 78.9 78.9 68.4 73.7 36.8 474A 47.4 47 A 


Reporting standards not met. There are too few cases for a reliable estimate. 
1These characteristics are based on self-reports provided by interview participants. 


NOTE: Response behavior was determined based on participants’ reaction to each example mailing. In the small number of cases where more than one household member participated in the interview, engagement was based on the primary interview participant's reaction to the mailing. 
Participants who "would engage" are those who stated they would open, read, or keep the mailing. The total number of interview participants was 85. Sample sizes are rounded to the nearest 5. Percentages are rounded to one decimal place but have not been changed to reflect sample size 


rounding. Details may not sum to totals due to rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table A.5.1. Percentage distribution of NHES:2019 screener mailing recall, by 


screener mailing: 2019 


Screener mailing Recalls Does not recall Unclear 

At least one mailing 76.5 21.2 2.4 
Initial screener package 60.0 36.5 3.5 
Pressure-sealed envelope 41.2 47.1 11.8 
Second screener package 35.3 52.9 11.8 
Third screener package 48.2 45.9 5.9 
Fourth screener package 36.5 54.1 9.4 
FedEx package! 36.5 54.1 9.4 
First Class mail package 35.3 52.9 11.8 


INHES:2019 included an experiment that varied which of the reminder mailings was sent via FedEx. Some participants were sent the second screener 
package via FedEx and others were sent the fourth screener package via FedEx. They were sent the other of these two packages by First Class mail. The 
estimates reported here show the recall of the FedEx or First Class mail reminder regardless of the timing of receipt. 

NOTE: Percentages are based on interview participants’ self reports during an activity where they were shown the NHES:2019 screener mailings. It is 
possible that additional participants received the mailings but either did not recall doing so or did not mention it during the interview. It is also possible 
that a different household member would have recalled the mailing. In the small number of cases where more than one household member participated in 
the interview, recall is based on the primary interview participant's recall of the mailing. Rounded number of eligible interview participants is 85. Details 
may not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


Table A.7.1. Number and percentage distribution of observation outcome for 
nonrespondent addresses sampled for observation component: 2019 


Number of Percentage 
Observation outcome addresses of addresses 
Total 590 100.0 
Observed 450 76.1 
Partially observed, could observe the exterior of the multi-unit 
building but not the sampled unit? 100 16.8 
Not observed, could not access address2 20 2.5 
Not observed, could not locate address3 20 2.9 
Not observed, other reason* 10 1.7 


1Partially observed addresses are those where the observer was able to observe the exterior of the multi-unit building but was not able to enter the 
building to observe the interior entry to the sampled unit. 

2Addresses that could not be accessed are those where observers could not get close enough to confirm whether the address existed (for example, if the 
address was in a gated community to which the observer could not gain access). 

3Addresses that could not be located are those where the observer was able to access the location where the address should have been, but the observer 
could not find any evidence that the address existed. 

4Addresses that were not observed for another reason were those where the observer could not make the observation for reasons other than not being 
able to access or locate the address (for example, because the observer was concerned about safety). 

NOTE: Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing 
were excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but have not been changed 
to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table A.7.2. Percentage distribution of observation outcome for 
nonrespondent addresses sampled for observation component, 
by selected characteristics: 2019 


Number of 
Selected characteristics addresses 
Total 590 
NHES:2019 screener mailings language 
All bilingual mailings (English and Spanish) 230 
Mix of bilingual mailings and English-only 290 
mailings 
All English-only mailings 70 
Age of head of household2 
18-34 50 
35-44 80 
45-54 90 
55-64 80 
65 and older 70 
Missing 220 
Gender of head of household2 
Male 180 
Female 180 
Missing 240 
Education of head of household? 
Less than high school credential 130 
High school credential 90 
Some college 120 
Bachelor's degree 70 
Graduate degree 40 
Missing 130 
Race of head of household? 
White, non-Hispanic 190 
Black, non-Hispanic 100 
Hispanic 130 
Other race, non-Hispanic 40 
Missing 140 
Household income2 
Less than $50,000 250 
$50,000-$74,999 70 
$75,000-$99,999 60 
$100,000 or higher 140 
Missing 70 
Household flagged as having children? 
Yes 140 
No 450 


Observation outcome! 


Observed 
76.1 


74.7 
75.6 
83.1 


79.6 
82.7 
90.2 
81.5 
80.0 
63.4 


79.3 
83.4 
68.1 


83.1 
80.9 
86.3 
75.7 
85.4 
53.4 


81.8 
80.8 
85.5 
80.6 
54.4 


74.4 
89.6 
85.0 
83.1 
48.6 


91.5 
71.2 


Partially 
observed 


16.8 


16.7 
17.5 


13.9 


16.7 
12.4 

6.5 
12.4 
12.3 
25.9 


13.4 
10.9 
23.8 


10.0 
12.4 
8.1 
18.9 
7.3 
36.6 


13.4 
9.1 
7.6 

19.4 

35.3 


16.7 

9.0 
13.3 
11.3 
37.8 


5.0 
20.5 


Not 
observed 


7.1 


8.6 
6.9 


10.7 


Chi-square 
statistic 


Tt 
3.13 


34.39 * 


16.07 * 


57.58 * 


57.42 * 


46.82 * 


24.61 * 


See notes at end of table. 
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Table A.7.2. Percentage distribution of observation outcome for nonrespondent 


addresses sampled for observation component, by selected 
characteristics: 2019—Continued 


Observation outcome! 


Number-——————_____—_—+___ 
of Partially Not 
Selected characteristics addresses Observed observed observed 
Number of adults in household2 
1 adult 330 74.4 19.3 6.3 
2 adults 130 88.5 4.6 6.9 
3 adults or more 50 94.3 $ $ 
Missing 70 48.6 37.8 13.5 
Phone number available2 
Yes 360 82.6 10.5 6.9 
No 230 65.6 26.9 7.5 
Route type? 
Street 370 91.3 2.7 6.0 
High rise 220 51.1 39.9 9.0 
Dwelling type? 
Single-unit 340 92.9 1.5 5.6 
Multi-unit 250 53.0 37.8 9.2 
Home tenure2 
Own 290 88.8 5.8 5.4 
Rent 200 71.3 22.1 6.7 
Missing 100 48.0 39.0 13.0 
Urbanicity2 
Urban 270 67.9 24.6 7.5 
Suburban 290 83.7 11.1 5.2 
Town + + + + 
Rural + + + + 
Region2 
Northeast 80 79.3 14.6 6.1 
South 250 78.1 13.4 8.5 
Midwest 170 73.8 20.4 5.8 
West 90 71.6 21.6 6.8 
Race/ethnicity stratum’ 
25% or more Black 150 75.2 17.7 7.2 
40% or more Hispanic 170 77.2 15.8 7.0 
Other 270 75.9 17.0 7.2 
Tract poverty rate? 
Less than 20% 390 78.1 14.8 7.1 
20% or more 200 72.1 20.8 7.1 


Chi-square 
statistic 


56.96 * 


27.60 * 


145.00 * 


145.37 * 


77.00 * 


39.78 * 


6.26 


0.23 


3.44 


See notes at end of table. 
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Table A.7.2. Percentage distribution of observation outcome for 
nonrespondent addresses sampled for observation component, 
by selected characteristics: 2019—Continued 


Selected characteristics 
Percent of households in Census block that 
include a child? 
First quartile 
Second quartile 
Third quartile 


Fourth quartile 
Percent of persons in Census block that 
speak a language other than English? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Percent of persons in Census block without a 
high school diploma or the equivalent? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Low Response Score* 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Residential high-speed internet per 1000 
households® 
600 or less 
601-800 


801 or more 


tNot applicable. 
#Rounds to zero. 


+Reporting standards not met. There are too few cases for a reliable estimate. 


*p<0.05 


Number. 


of 


addresses 


160 
120 
140 
170 


50 
100 
180 
260 


140 
110 
130 
210 


110 
100 
130 
250 


140 
220 
230 


Observation outcome! 


Observed 


64.1 
83.2 
76.6 
81.5 


84.0 
74.7 
76.4 
74.8 


69.5 
83.3 
75.0 
77.4 


91.1 
86.3 
70.5 
68.3 


74.1 
72.2 
80.8 


Partially Not 
observed _ observed 


28.9 7.1 
13.5 3.4 
10.6 12.8 
13.3 5.2 
10.0 6.0 
16.8 8.4 
15.4 8.2 
19.1 6.1 
18.4 12.1 
13.9 2.8 
19.5 5.5 
15.6 7 

$ Fa 
10.5 3.2 
23.3 6.2 
22.6 9.1 
17.3 8.6 
20.4 7A 
13.3 6.0 


Chi-square 
statistic 


32.68 * 


3.91 


11.31 


36.31 * 


5.48 


1Partially observed addresses are those where the observer was able to observe the exterior of the multi-unit building, but was not able to 
enter the building to observe the interior entry to the sampled unit. Addresses that were not observed include those where observers could 
not get close enough to confirm that the address existed (for example, if the address was ina gated community); addresses that could not be 
located (the observer was able to access the location where the address should have been, but the observer could not find any evidence that 
the address existed); and addresses that were not observed for another reason (for example, the observer was concerned about safety). 
2These characteristics are based on variables available on the NHES:2019 sampling frame. 
3These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with 


the lowest prevalence of the characteristic in question and those that are in the fourth quartile are t 
4The Low Response Score is a derived variable that identifies block groups with characteristics asso 


hose with the highest prevalence. 
ciated with low mail return rates to the 


2010 Decennial Census. A higher score corresponds to a lower expected mail return rate. This variable was not available for a very small 


number of cases, and these cases have been excluded from this analysis. 
5This characteristic is based on tract-level estimates of Internet penetration provided by the Federa 
NOTE: Addresses that ended up responding to NHES:2019 and addresses that had at least one unde 


Communications Commission (FCC). 
iverable as addressed (UAA) NHES:2019 


mailing were excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but have 


not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household 
2019; U.S. Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennia’ 


Communications Commission (FCC), 2017. 


Education Surveys Program (NHES), 
Census, 2010; and Federal 
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Table A.7.3. Wald joint significance tests from multinomial logistic regression 


predicting observability, by independent variables: 2019 


Independent variables 

NHES:2019 bilingual screener mailings status 
Age of head of household! 

Gender of head of household! 

Education of head of household! 

Race of head of household! 

Household income! 

Whether household flagged as having children 
Number of adults in household! 

Whether phone number available! 

Route type! 

Dwelling type 

Home tenure! 

Urbanicity! 

Region! 

Race/ethnicity stratum2 

Tract poverty rate2 


Percent of households in Census block that include a child2 
Percent of persons in Census block that speak a language other than English? 
Percent of persons in Census block without a high school diploma or the equivalent? 


Low Response Score? 


Residential high-speed internet per 1000 households* 


*p <.05. 


1These variables are available on the NHES:2019 sampling frame. 
2These characteristics are based on American Community Survey (2013-2017) five-year estimates. 


Wald test statistic 
2.81 
5.78 
3.65 

12.17 
12.27 
10.53 
3.82 
1.75 
9.09 * 
7.95 * 
7.83 * 
3.30 
16.90 * 
25.05 * 
2.97 
6.03 * 
16.06 * 
6.02 
11.43 
11.52 
6.52 


3The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 
Decennial Census. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

4This variable is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The dependent variable is a collapsed version of the observability variable with three categories—observed, partially observed, and not 
observed. "Not observed" is the reference category. Missing values on the independent variables are treated as a separate category. Addresses that 
ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from 


this analysis. The pseudo R2 value for the model is 0.43. The rounded sample size is 590. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission 


(FCC), 2017. 
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Table A.7.4. Number and percentage distribution of residential occupancy status 
observation for nonrespondent addresses sampled for observation 
component: 2019 


Residential occupancy status Number of addresses Percentage of addresses 

Total 550 100.0 
Occupied residential unit 400 72.6 
Residential unit, could not determine occupancy status 140 25.8 
Vacant residential unit! 10 1.5 
Vacant lot $ t 


Commercial business # # 


#Rounds to zero. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

1Vacant residential unit includes: (1) temporarily vacant addresses where the unit had a for sale or for rent sign or a lock box and there was no 
evidence of current occupants, (2) seasonally vacant addresses where the unit was located ina resort area and was well-maintained enough to suggest 
that it had tenants during other parts of the year, but there were no evidence of current occupants, and (3) permanently vacant addresses where the 
address appeared uninhabitable (e.g., boarded windows/doors, holes in the walls or roof), was condemned or was under construction. 

NOTE: Residential occupancy status was not collected for addresses that were not able to be observed (for example, for addresses that could not be 
located). Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing 
were excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but have not been changed 
to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table A.7.5. Agreement rate between frame and observation variables for 
addresses with inconsistent undeliverable as addressed (UAA) 
NHES:2019 mailing outcomes, by selected characteristics: 2019 


Selected characteristics 


Agreement rate 


Occupancy status 

Total 82.1 
Occupancy status on frame1 

Vacant2 50.0 

Occupied 87.9 
Occupancy status observation? 

Vacant? 42.9 

Occupied 90.6 
Structure type 

Total 86.8 
Structure type on frame1 

Single-unit 68.2 

Multi-unit 95.7 
Structure type observation? 

Single-unit 88.2 

Attached*+ 40.0 

Apartment® $ 
Presence of children 

Total 88.1 
Presence of children on frame1 

Yes + 

No or unknown 92.1 
Presence of children observation? 

Yes 40.0 

No or unclear 94.6 
Household income® 

Total 48.1 
Household income on frame! 

Bottom third 52.0 

Middle third 66.7 

Top third 16.7 
Household income observation3 

Bottom third 59.1 

Middle third 38.5 

Top third 50.0 


Reporting standards not met. There are too few cases for a reliable estimate. 
1These characteristics are based on variables available on the NHES:2019 sampling frame. 


2For frame data, vacant addresses were those flagged as being vacant. For observed data, vacant addresses include those that were seasonally, 


temporarily, and permanently vacant. 


3These characteristics are based on address observations collected as part the qualitative nonresponse study. 
4Attached structures include duplexes, townhouses, and rowhouses. 
5Apartment structures include low-, medium-, and high-rise apartments. 


6For frame data, the bottom third includes those addresses with household incomes less than $50,000; the middle third includes those addresses with 


household incomes $50,000 to $99,999; the top third includes those addresses with household incomes of 
based ona combination of the distribution of household income in the United States in 2019 and the pre-ex 
sampling frame. For observed data, observers were asked to provide their best estimate of whether the add 
third, middle third, or top third of as compared to other households across the United States. 


100,000 or more. These cutoff points were 
isting income ranges available on the NHES 
ress's household income was in the bottom 


NOTE: The agreement rate is the percentage of addresses observed to have the same characteristic as found in the frame. Each analysis in this table is 


limited to cases that had data available from both sources. Rounded number of eligible cases is 70 for struc 
presence of children, and 50 for household income. Details may not sum to totals because of rounding. 


ure type, 40 for occupancy status, 40 for 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019 
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Appendix B. Additional Qualitative Nonresponse Study Methodology 


Appendix B. Additional Qualitative Nonresponse Study 
Methodology 
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Appendix B. Additional Qualitative Nonresponse Study Methodology 


This appendix includes additional information about the NHES:2019 qualitative 
nonresponse study methodology that readers should be aware of when interpreting the 
results. For more information on the NHES:2019 data collection activities and sampling 
methods, see the National Household Education Surveys Program of 2019: Data File User’s 
Manual (Jackson et al. 2021). 


B.1 Sampling 
B.1.1 Site Selection 


To maximize efficiency, data collection for the qualitative nonresponse study was restricted 
to seven 30-mile radius sites across the United States. Several goals guided the selection of 
these sites: (1) maximizing the number of available nonresponding addresses, (2) having 
regional diversity in the selected sites, and (3) ensuring sufficient representation of specific 
types of addresses. For example, the site selection aimed to maximize representation of likely 
Hispanic addresses, which have traditionally had lower NHES response rates. In addition, it 
was important to be able to sample addresses located in a variety of locale types (i-e., urban, 
suburban, rural), since households in different types of locales likely face different types of 
mail delivery challenges. 


Qualitative interview sites 


Address observations were conducted in all seven sites, and qualitative interviews were 
conducted in four of the sites. To select the qualitative interview sites, NHES:2016 response 
data was used because the NHES:2019 collection had not begun at the time this decision 
needed to be made. The ABS sampling methods used for NHES:2019 were nearly identical to 
those used for NHES:2016, and the mailing protocol for the 2016 web-push condition was 
very similar to the NHES:2019 web-push protocol. Therefore, it was believed that the 
NHES:2016 response data could be used as a proxy. 


To facilitate site selection, the NHES:2016 mixed mode sample was subset to the addresses 
that had not yet responded prior to the fourth screener package and were not P.O. boxes, 
drop points, or rural routes. These addresses were mapped onto a GIS shapefile of U.S. cities 
and towns. Exhibit B.1 below plots the location of these NHES:2016 screener 
nonrespondents. A starting set of potential cities and towns that could serve as the center 
points of the study sites was identified by applying a series of eligibility rules; any city or 
town that did not have at least 50 eligible NHES:2016 nonresponding addresses within a 30- 
mile radius were excluded from consideration, as was any city or town that did not meet a 
region-specific target for the number of nonresponse-study-eligible nonrespondent 
addresses for each high-priority subgroup (Hispanic addresses, addresses with children, and 
addresses whose head of household has less than a high school degree). The three cities that 
had already been designated as “observation-only” sites (described below) were excluded as 


1 A threshold of 50 was used because these are the areas expected to have at least 100 nonresponse-study-eligible nonrespondents in 
NHES:2019 based on the proportion of sampled NHES:2016 mixed-mode cases relative to the number of addresses sampled for the 
NHES:2019 baseline web-push protocol. 
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possible interview sites. Cities and towns whose 30-mile radius overlapped with the 
observation-only sites also were excluded to make the sample as regionally diverse as 
possible. 


Exhibit B.1. Location of NHES:2016 screener nonrespondents 


al 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National 
Household Education Surveys Program (NHES), 2016. 


Using the remaining cities and towns after these exclusions were applied, a k-means 
clustering algorithm was used to divide the remaining cities and towns into four clusters. 
The following variables were used in the clustering algorithm:2 


e Percent with Hispanic head of household 

e Percent with Black head of household 

e Percent with household income under $25,000 

e Percent in which the head of household does not have a high school diploma 
e Percent in which the head of household is under 35 

e Percent urban 


From each of the four clusters, the city that was closest to the cluster centroid was selected. 
Because the k-means algorithm includes some randomization, the procedure was run five 
times, each using a different random seed; this led to five possible sets of four sites each. 
Each set of sites was examined to identify the set with the greatest diversity in terms of the 


2 K-means aims to minimize within-cluster variation while maximizing between-cluster variation with respect to specified variables of 
interest. 
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size of the selected metro area and the demographic characteristics of the sites, particularly 
racial composition, socioeconomic status, and urbanicity. 


Of the five sets of sites produced by the algorithm, the fourth set was selected because it 
seemed the most geographically diverse, included one city from each Census Region, and met 
the criteria outlined above. More specifically, the center points of the sites were in 
Connecticut, Ohio, Texas, and California. Because the sites each had a 30-mile radius, some 
of them included addresses in more than one state. 


Observation sites 


Observations were conducted in seven sites. In addition to the four sites described above, 
three observation-only sites were selected. These sites were selected because of their 
proximity to AIR offices, which increased the efficiency of the address observation fieldwork 
efforts. They also complemented the characteristics of the other four sites. The center points 
of these sites were in Texas, Illinois, and Washington, DC. Again, because the sites each had 
a 30-mile radius, some of them included addresses in more than one state. 


B.1.2 Address Selection 


The address sample design was driven by three priorities of (1) sampling 100 addresses per 
qualitative site and 120 addresses per observation-only site; (2) achieving sufficient 
qualitative interviews from Hispanic households, households with children, and low- 
education households to allow for separate analyses of these key subgroups; and (3) 
ensuring that the sample was reasonably representative of all nonresponse-study-eligible 
nonrespondent addresses located at the seven selected sites. 


Address selection at qualitative interview sites 


At the four qualitative sites, addresses needed to meet the following criteria to be eligible for 
sampling for the qualitative study: 


e Lie within a 30-mile radius of the site’s center, as defined by latitude and longitude 


e Bepresent on the data collection operations mailout label file for the final NHES:2019 
screener nonresponse follow-up mailing and not be on the pull file for this mailing* 


e Bepart ofthe updated web-push mailing protocol treatment group, excluding the opt- 
out screener materials treatment group® 


3 These priorities were competing because it was necessary to oversample the smaller sites to achieve a uniform sample size per site, and 
to oversample the key subgroups for the qualitative study to achieve enough interviews for analysis; but the more variance there was in 
sampling rates across subgroups and sites, the less the sample would resemble the eligible population. 

4 The label file is created approximately two weeks before mailout and contains addresses that are set to receive the specified mailing 
wave. The pull file is created a few days before mailout and contains addresses that responded in the interim and thus are to be “pulled” 
from the mailing. 

5 The NHES:2019 included multiple methodological experiments that impacted exactly what materials respondents received with each 
mailing. Detailed descriptions of the experimental treatments can be found in Medway et al. (forthcoming). 
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e Not beadrop point address, P.O. box, or rural route address 


e Have a status code on the data collection operations mailout label file for the final 
NHES:2019 screener nonresponse follow-up mailing indicating that the case was not 
a hard refusal or an ineligible address 


A total of 400 addresses were selected, 100 from each site. Within each site, eligible 
addresses were divided into eight strata, defined by crossing the sampling frame indicators 
for Hispanic status, whether the household has children, and whether the head of household 
has a high school diploma. 


The sample size for each stratum was chosen using a constrained optimization routine that 
minimized the variation in sampling rates across strata and sites while requiring at least 120 
addresses to be sampled from each key subgroup (Hispanics, households with children, and 
low-education households) across the four sites. Sampling 120 addresses from each 
subgroup increased the likelihood of achieving enough interviews with that group, allowing 
some safety margin for inaccuracies in the sampling frame flags and for differential 
nonresponse to the qualitative interviews. Since weights were not constructed for the 
qualitative nonresponse study, the oversample required to meet these targets implied that 
the qualitative sample would not be purely “representative” of the eligible addresses. The 
requirement to sample exactly 100 addresses from each site implied a further departure 
from representativeness, since the number of eligible addresses varied across sites. 
However, by minimizing the variation in sampling rates, the constrained optimization 
routine helped to meet these targets with the lowest possible sampling bias. 


Within each stratum, a systematic random sample was selected after sorting on an indicator 
for multi-unit dwellings and then on the nine-digit ZIP code. Sorting achieves implicit 
stratification, helping to ensure that multi-unit addresses made up a similar proportion of 
the sample as of all eligible addresses, and that the geographic distribution of the sample 
mirrored that of eligible addresses. 


Address selection at observation-only sites 


At the three observation-only sites, the eligibility criteria were the same as for the qualitative 
sites, with one exception. Addresses with a UAA status code on the data collection operations 
mailout label file for the final NHES:2019 screener nonresponse follow-up mailing were 
eligible for the observation-only study if at least one of the first three mailings was not 
returned as undeliverable. The inclusion of these “inconsistent UAAs” allowed the 
observations to gather insight into the characteristics of addresses with such outcomes. 


Within each site, eligible addresses were divided into two strata: inconsistent UAAs and non- 
UAAs. A total of 120 addresses—100 non-UAAs and 20 partial UAAs—were selected from 
each site. A systematic random sample was drawn from each stratum, after sorting on a 
Hispanic indicator, a presence of children indicator, a low-education indicator, a multi-unit 
dwelling indicator, and the nine-digit ZIP code. Thus, although the key subgroups were not 
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oversampled from the observation-only sites, implicit stratification helped ensure that each 
group’s prevalence in the sample was similar to its prevalence among eligible addresses. 


B.1.3 Sample Representativeness 


The sample for this study was not designed to be nationally representative of NHES 
nonrespondents. The study was meant to provide theory building information and should 
not be interpreted as providing nationally representative official estimates. As noted above, 
the seven study sites were chosen non-randomly, and (due to the requirement to have at 
least 100 nonrespondents within a 30-mile radius) were in predominantly urban or 
suburban areas. Some sample bias is expected due to the oversampling of certain subgroups 
and differential sampling rates across sites (i.e., smaller sites required higher sampling rates 
to achieve a uniform sample size across sites). 


To provide readers with additional context about how well the sampled addresses 
represented the larger set of nonrespondents to NHES:2019, we compared the addresses 
sampled for the qualitative nonresponse study to those of NHES:2019 nonrespondents in 
terms of address-level and area-level variables on the NHES:2019 sampling frame and 
NHES:2019 paradata. For each characteristic, the absolute bias was calculated as the 
absolute value of the difference between sampled addresses and all nonrespondent 
addresses. 


Two analyses were conducted that focused on nonrespondent addresses: the first examined 
the characteristics of the 700 non-UAA addresses sampled for the observation component of 
study, while the second focused just on the 400 addresses sampled for the qualitative 
interview component of the study (see tables B.1 and B.2). Both of these analyses suggest the 
nonrespondent study sample was reasonably representative of NHES:2019 nonrespondents. 
In general, the characteristics showing the largest bias either were explicitly oversampled 
(e.g, households with children and low-education households) or are correlated with 
oversampled characteristics (e.g., low-income households and renters). In addition, as to be 
expected, there was some bias for characteristics that were influenced by the clustered 
design and the selection of somewhat more densely populated sites (e.g., locale, region, route 
type, dwelling type). The addresses sampled for the qualitative nonresponse study were 
generally less likely to be missing information on the sampling frame. 


Finally, a third analysis compared the characteristics of the inconsistent UAA addresses 
sampled for the qualitative nonresponse study and the larger set of addresses with 
inconsistent UAA outcomes for the NHES:2019 screener mailings (see table B.3). The 
patterns discussed above for nonrespondent addresses generally held for these addresses 
as well. In addition, among the inconsistent UAA addresses, the estimated bias was generally 
larger. 
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Table B.1. Number and percentage distribution of household-level auxiliary data for nonrespondent addresses and inconsistent UAA addresses, by 
qualitative nonresponse study sampling status: 2019 


Nonrespondent addresses! Inconsistent undeliverable as addressed (UAA) addresses2 
All nonrespondent Addresses sampled for observation Addresses sampled for interview All inconsistent UAA Addresses sampled for observation 
addresses component component addresses component 
Number of Percentage Numberof Percentage Absolute Numberof Percentage Absolute Numberof Percentage Number of Percentage Absolute 

Household-level auxiliary data? addresses of addresses _addresses _ of addresses bias* addresses _ of addresses bias> addresses ofaddresses addresses _ of addresses bias® 

Total 78,330 100.0 700 100.0 t 400 100.0 t 17,930 100.0 60 100.0 t 
Age of head of household 

18-34 7,980 10.2 60 8.9 1.3 40 9.5 0.7 1,400 7.8 5 5.0 2.8 

35-44 10,690 13.7 100 14.3 0.6 60 15.8 2.1 1,440 8.0 5 5.0 3.0 

45-54 11,340 14.5 110 15.9 1.4 60 14.8 0.3 1,330 7.4 10 15.0 7.6 

55-64 9,730 12.4 90 13.1 0.7 60 15.3 2.8 1,250 7.0 # # t 

65 and older 9,560 12.2 90 12.6 0.4 50 13.5 1.3 2,280 1257 15 21.7 8.9 

Missing 29,030 37.1 250 35.3 1.8 130 31.3 5.8 10,220 57.0 30 50.0 7.0 
Gender of head of household 

Male 23,920 30.5 220 31.1 0.6 130 32.3 1.7 3,860 21.5 20 30.0 8.5 

Female 22,210 28.4 200 28.7 0.4 130 31.3 2.9 3,940 22.0 20 30.0 8.0 

Missing 32,200 41.1 280 40.1 1.0 150 36.5 4.6 10,140 56.5 25 40.0 16.5 
Education of head of household 

Less than high school credential 10,640 13.6 150 21.6 8.0 120 30.0 16.4 1,740 9.7 5 10.0 0.3 

High school credential 16,290 20.8 110 15.7 5.1 70 17.8 3.1 2,670 14.9 15 23.3 8.4 

Some college 16,550 21.1 140 19.9 1.3 70 17.8 3.4 2,580 14.4 15 21.7 7.3 

Bachelor's degree 8,590 11.0 90 13.3 2.3 50 12.5 1.5 1,210 6.8 5 8.3 1.6 

Graduate degree 5,100 6.5 50 7A 0.9 20 5.5 1.0 800 4.5 5 5.0 0.5 

Missing 21,160 27.0 160 22.1 4.9 70 16.5 10.5 8,930 49.8 20 31.7 18.1 
Race/ethnicity of head of household 

White, non-Hispanic 30,040 38.4 230 33.0 5.4 150 36.3 2.1 4,990 278 20 33.3 5.5 

Black, non-Hispanic 10,590 13.5 110 16.0 2.5 60 13.8 0.2 2,170 12.1 10 18.3 6.2 

Hispanic 12,030 15.4 160 22.4 7.1 110 26.3 10.9 1,270 7.1 5 11.7 4.6 

Other race, non-Hispanic 3,030 3.9 40 5.6 1.7 30 6.3 2.4 330 1.9 # # Tt 

Missing 22,630 28.9 160 23.0 5.9 70 17.5 11.4 9,170 51.1 20 33.3 17.8 


See notes at end of table. 
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Table B.1. Number and percentage distribution of household-level auxiliary data for nonrespondent addresses and inconsistent UAA addresses, by 


qualitative nonresponse study sampling status: 2019—Continued 


All nonrespondent 


Nonrespondent addresses! 
Addresses sampled for observation 


Addresses sampled for interview 


Inconsistent undeliverable as addressed (UAA) addresses? 
All inconsistent UAA Addresses sampled for observation 


addresses component component addresses component 
Number of Percentage Numberof Percentage Absolute Numberof Percentage Absolute Numberof Percentage Number of Percentage Absolute 

Household-level auxiliary data? addresses of addresses addresses _ of addresses biast | addresses of addresses bias> addresses ofaddresses addresses of addresses bias® 
Household income 

Less than $50,000 31,460 40.2 280 40.3 0.1 190 48.0 7.8 6,520 36.4 20 33.3 3.0 

$50,000-$74,990 10,150 13.0 80 11.7 1.2 50 11.5 1.5 1,540 8.6 5 10.0 1.4 

$75,000-$99,999 7,930 10.1 80 11.6 1.5 40 10.5 0.4 1,000 5.6 5 11.7 6.1 

$100,000 or higher 14,280 18.2 170 23.6 5.3 90 21.3 3.0 1,850 10.3 10 20.0 9.7 

Missing 14,520 18.5 90 12.9 5.7 40 8.8 9.8 7,030 39.2 15 25.0 14.2 
Household flagged as having children 

Yes 15,030 19.2 180 25.1 6.0 120 30.0 10.8 1,870 10.4 5 6.7 3.8 

No 63,300 80.8 520 74.9 6.0 280 70.0 10.8 16,060 89.6 55 93.3 3.8 
Number of adults in household 

1 adult 41,500 53.0 390 55.4 2.5 230 58.3 5.3 8,530 47.6 35 60.0 12.4 

2 adults 16,480 21.0 160 23.4 24 100 24.8 3.7 1,900 10.6 10 16.7 6.1 

3 or more adults 5,920 7.6 60 8.3 0.7 30 8.3 0.7 490 2.7 # # t 

Missing 14,430 18.4 90 12.9 5.6 40 8.8 9.7 7,010 39.1 15 23.3 15.8 
Phone number available 

Yes 47,540 60.7 440 63.1 2.5 270 68.3 7.6 8,140 45.4 25 41.7 3.8 

No 30,800 39.3 260 36.9 2.5 130 31.8 7.6 9,780 54.6 35 58.3 3.8 
Route type’ 

Street 55,560 70.9 440 62.3 8.6 270 68.5 2.4 10,860 60.6 30 50.0 10.6 

High rise 22,160 28.3 260 37.7 9.4 130 31.5 3.2 6,820 38.0 30 50.0 12.0 

P.O. Box 570 0.7 t t t t t t 230 1.3 t t t 

Rural 40 0.1 t t t t t t 20 0.1 t t t 


See notes at end of table. 


B-8 


Appendix B. Additional Qualitative Nonresponse Study Methodology 


Table B.1. Number and percentage distribution of household-level auxiliary data for nonrespondent addresses and inconsistent UAA addresses, by 
qualitative nonresponse study sampling status: 2019—Continued 


Nonrespondent addresses! Inconsistent undeliverable as addressed (UAA) addresses? 
All nonrespondent Addresses sampled for observation Addresses sampled for interview All inconsistent UAA Addresses sampled for observation 
addresses component component addresses component 


Number of Percentage Numberof Percentage Absolute Numberof Percentage Absolute Numberof Percentage Number of Percentage Absolute 


Household-level auxiliary data3 addresses _ of addresses addresses _ of addresses bias+ addresses _ of addresses bias5 addresses ofaddresses addresses _ of addresses bias® 
Dwelling type 
Single-unit 53,160 67.9 410 58.3 9.6 260 64.8 3.1 10,290 57.4 25 40.0 17.4 
Multi-unit 24,600 31.4 290 41.7 10.3 140 35.3 3.8 7,410 41.3 35 60.0 18.7 
Missing 570 0.7 # # Tt # # T 230 1.3 # # Tt 
Home tenure 
Own 41,150 52.5 360 51.0 1.5 200 50.0 2.5 6,040 33.7 25 38.3 4.6 
Rent 21,130 27.0 220 32.0 5.0 150 37.3 10.3 4,840 27.0 15 25.0 2.0 
Missing 16,050 20.5 120 17.0 3.5 50 12.8 7.8 7,050 39.3 20 36.7 2.6 
tNot applicable. 


#Rounds to zero. 

1Nonrespondent addresses were those that met both of these criteria: (1) Their final NHES:2019 final screener was nonrespondent. (2) They did not have any UAA outcomes for the screener mailings. 
2Inconsistent UAA addresses were those that had a UAA for some, but not all, of the NHES:2019 screener mailings. 

3The household-level auxiliary data are based on variables available on the NHES: 2019 sampling frame. 

4Absolute bias is the absolute value of the difference between all nonrespondent addresses and addresses sampled for the observation component. 

5Absolute bias is the absolute value of the difference between all nonrespondent addresses and addresses sampled for the interview component. 

6Absolute bias is the absolute value of the difference between all inconsistent UAA addresses and UAA addresses sampled for the observation component. 

7P.0. Box and rural route addresses were not eligible to be sampled for the qualitative nonresponse study. 

NOTE: Sample sizes are rounded to the nearest 10, except for inconsistent UAA addresses sampled for the observation component (rounded to the nearest 5 due to small sample sizes). Percentages are rounded to one decimal place but have not been changed to reflect sample size 
rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table B.2. Number and percentage distribution of area-level auxiliary data for nonrespondent addresses and inconsistent UAA addresses, by qualitative 
nonresponse study sampling status: 2019 


Nonrespondent addresses! Inconsistent undeliverable as addressed (UAA) addresses? 
All nonrespondent Addresses sampled for observation Addresses sampled for interview All inconsistent UAA Addresses sampled for observation 
addresses component component addresses component 


Numberof Percentage Numberof Percentage Absolute Numberof Percentage Absolute Numberof Percentage Numberof Percentage Absolute 


Area-level auxiliary data addresses of addresses addresses of addresses bias? addresses __ of addresses bias* addresses _ofaddresses addresses _ of addresses bias® 

Total 78,330 100.0 700 100.0 t 400 100.0 t 17,930 100.0 60 100.0 t 
Urbanicity® 

Urban 31,590 40.3 310 44.7 4.4 165 41.3 0.9 7,690 42.9 25 43.3 0.4 

Suburban 28,930 36.4 350 49.7 13.3 216 54.0 17.6 5,290 29.5 30 48.3 18.8 

Town 6,780 8.7 10 1.0 7.7 # # t 2,360 13.2 £ £ t 

Rural 11,430 14.6 30 4.6 10.0 16 4.0 10.6 2,590 14.4 5 6.7 7.8 
Region® 

Northeast 13,200 16.9 100 14.3 2.6 100 25.0 8.1 2,580 14.4 # # t 

South 33,720 43.0 300 42.9 0.2 100 25.0 18.0 8,130 45.4 40 66.7 21.3 

Midwest 13,890 17.7 200 28.6 10.8 100 25.0 73 4,120 23.0 20 33.3 10.4 

West 17,530 22.4 100 14.3 8.1 100 25.0 2.6 3,100 17.3 # # t 
Race/ethnicity stratum” 

25% or more Black 18,890 24.1 170 24.7 0.6 70 18.3 5.9 5,160 28.8 20 36.7 7.9 

40% or more Hispanic 15,450 19.7 190 27.6 7.8 130 31.8 12.0 2,640 14.7 15 21.7 6.9 

Other 43,990 56.2 330 47.7 8.4 200 50.0 6.2 10,120 56.5 25 41.7 14.8 
Tract poverty rate” 

Less than 20% 49,530 63.2 470 67.4 4.2 250 62.0 1.2 10,000 55.8 35 61.7 5.9 

20% or more 28,800 36.8 230 32.6 4.2 150 38.0 1.2 7,930 44.2 25 38.3 5.9 
Percent of households in Census block 

that include a child’ 

First quartile 17,870 22.8 190 26.4 3.6 90 23.3 0.4 5,720 31.9 20 30.0 1.9 

Second quartile 18,510 23.6 140 19.4 4.2 90 21.8 1.9 4,420 24.7 15 21.7 3.0 

Third quartile 19,500 24.9 180 25.6 0.7 100 24.8 0.2 3,990 22.3 15 23.3 1.0 

Fourth quartile 22,410 28.6 200 28.6 0.1 120 30.3 1.6 3,770 21.1 15 25.0 3.9 


See notes at end of table. 
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Table B.2. Number and percentage distribution of area-level auxiliary data for nonrespondent addresses and inconsistent UAA addresses, by qualitative 


nonresponse study sampling status: 2019—Continued 


Nonrespondent addresses! Inconsistent undeliverable as addressed (UAA) addresses? 
All nonrespondent Addresses sampled for observation Addresses sampled for interview All inconsistent UAA Addresses sampled for observation 
addresses component component addresses component 

Number of Percentage Numberof Percentage Absolute Number of Percentage Absolute Numberof Percentage Numberof Percentage Absolute 

Area-level auxiliary data addresses _ of addresses addresses _ of addresses bias? addresses of addresses bias* addresses _ of addresses addresses of addresses bias® 
Percent of persons in Census block that 

speak a language other than English’ 

First quartile 17,220 22.0 60 9.1 12.9 40 10.3 11.7 5,050 28.2 5 11.7 16.5 

Second quartile 17,150 21.9 120 17.1 4.8 80 19.5 2.4 4,420 24.7 10 15.0 9.7 

Third quartile 19,420 24.8 220 31.7 6.9 110 26.8 2.0 4,290 23.9 15 28.3 44 

Fourth quartile 24,520 31.3 290 42.0 10.7 170 43.5 12.2 4,160 23.2 25 45.0 21.8 


Percent of persons in Census block without 
a high school diploma or the equivalent’ 


First quartile 16,040 20.5 180 25.6 5.1 80 20.5 0.0 3,580 20.0 10 

Second quartile 17,460 22.3 130 19.1 3.2 80 19.5 2.8 3,940 22.0 15 

Third quartile 20,160 25.8 150 21.6 4.2 80 20.3 5.5 4,910 27.4 15 

Fourth quartile 24,630 31.5 240 33.7 2.3 160 39.8 8.3 5,500 30.7 20 
Low Response Score® 

First quartile 14,390 18.5 140 19.9 1.4 90 21.6 3.1 2,930 16.6 10 

Second quartile 17,490 22.5 120 17.0 5.4 70 17.5 4.9 3,630 20.5 5 

Third quartile 20,850 26.8 160 22.3 4.4 80 19.6 7.2 4,540 25.6 15 

Fourth quartile 25,180 32.3 290 40.8 8.5 170 41.4 9.0 6,620 37.4 30 
Residential high-speed internet per 1000 

households? 

600 or less 24,270 31.0 160 22.4 8.6 120 29.0 2.0 7,180 40.0 15 

601-800 28,360 36.2 250 36.0 0.2 150 37.5 13 5,860 32.7 15 

801 or more 25,700 32.8 290 41.6 0.2 130 33.5 0.7 4,900 27.3 30 

tNot applicable. 


#Rounds to zero. 

Reporting standards not met. There are too few cases for a reliable estimate. 

1Nonrespondent addresses were those that met both of these criteria: (1) Their final NHES:2019 final screener was nonrespondent. (2) They did not have any UAA outcomes for the screener mailings. 

2Inconsistent UAA addresses were those that had a UAA for some, but not all, of the NHES:2019 screener mailings. 

3Absolute bias is the absolute value of the difference between all nonrespondent addresses and addresses sampled for the observation component. 

4Absolute bias is the absolute value of the difference between all nonrespondent addresses and addresses sampled for the interview component. 

5Absolute bias is the absolute value of the difference between all inconsistent UAA addresses and UAA addresses sampled for the observation component. 

6These characteristics are based on variables available on the NHES: 2019 sampling frame. 

7These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest prevalence of the characteristic in question and those that are in the fourth quartile are those with the highest prevalence. 


16.7 
26.7 
26.7 
30.0 


13.3 
11.7 
21.7 
53.3 


28.3 
21.7 
50.0 


3.3 
4.7 
0.7 
0.7 


3.2 
8.8 
3.9 
16.0 


11.7 
11.0 
22.7 
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8The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial Census. A higher score corresponds to a lower expected mail return rate. This variable was not available for a very small number of cases, and 
these cases have been excluded from this analysis. 


°This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 


NOTE: Sample sizes are rounded to the nearest 10, except for inconsistent UAA addresses sampled for the observation component (rounded to the nearest 5 due to small sample sizes). Percentages are rounded to one decimal place but have not been changed to reflect sample size rounding. Details 
may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission 
(FCC), 2017. 
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Table B.3. Number and percentage distribution of data collection paradata for nonrespondent addresses and inconsistent UAA addresses, by qualitative 
nonresponse study sampling status: 2019 


Nonrespondent addresses! 


All nonrespondent 
addresses 


Addresses sampled for observation 


Number of Percentage Numberof Percentage Absolute 


Addresses sampled for interview 
component component 


Number of 
addresses 


Inconsistent undeliverable as addressed (UAA) addresses? 


All inconsistent UAA Addresses sampled for observation 


Total 78,330 100 700 100.0 t 


Data collection paradata addresses of addresses addresses _ of addresses bias3 
Nonresponse study site® 

Texas interview site 530 5.9 100 14.3 8.4 

Connecticut 1,910 21.2 100 14.3 6.9 

California 1,130 12.5 100 14.3 1.7 

Ohio 490 5.4 100 14.3 8.9 

Texas observation site 3,060 34.0 100 14.3 19.7 

Illinois 540 6.0 100 14.3 8.3 

Washington, DC 1,350 15.0 100 14.3 0.7 
NHES:2019 final screener response status 

Responded # # 80 12.0 t 

Did not respond 78,330 100.0 620 88.0 12.0 
NHES:2019 undeliverable as addressed (UAA) 

mailings status 

One or more UAA mailings # # 30 3.9 t 

No UAA mailings 78,330 100.0 670 96.1 t 
NHES:2019 screener mailings language 

All bilingual mailings (English and Spanish) 30,080 38.4 280 39.9 1.5 

Mix of bilingual mailings and English-only 

mailings 30,950 39.5 340 48.0 8.5 

All English-only mailings 17,310 22.1 90 12.1 10.0 

tNot applicable. 


#Rounds to zero. 


_____addresses_ component 

Percentage Absolute Numberof Percentage Numberof Percentage Absolute 

of addresses bias* addresses ofaddresses addresses _ of addresses bias 

400 100.0 t 17,930 100.0 60 100.0 t 
100 25.0 19.1 130 7.8 t t t 
100 25.0 3.8 500 31.1 t t t 
100 25.0 12.5 220 13.5 t t t 
100 25.0 19.6 90 5.4 tT t tT 
+t + t 300 18.4 20 33.3 14.9 
5 - + 160 9.7 20 33.3 23.6 
+t t + 230 14.0 20 33.3 19.4 
50 12.0 t 1,980 11.0 5 5.0 6.0 
350 88.0 12.0 15,950 89.0 55 95.0 6.0 
20 4.3 - 17,930 100.0 60 100.0 0.0 
380 95.7 4.3 # # # # t 
130 32.3 6.1 7,170 40.0 20 36.7 3.3 
210 52.8 13.3 5,690 31.8 30 50.0 18.3 
60 15.0 7A 5,060 28.2 10 13.3 14.9 


1Nonrespondent addresses were those that met both of these criteria: (1) Their final NHES:2019 final screener was nonrespondent. (2) They did not have any UAA outcomes for the screener mailings. 


2Inconsistent UAA addresses were those that had a UAA for some, but not all, of the NHES:2019 screener mailings. 

3Absolute bias is the absolute value of the difference between all nonrespondent addresses and addresses sampled for the observation component. 
4Absolute bias is the absolute value of the difference between all nonrespondent addresses and addresses sampled for the interview component. 
5Absolute bias is the absolute value of the difference between all inconsistent UAA addresses and UAA addresses sampled for the observation component. 


6The percentage of nonrespondent and inconsistent UAA addresses in the seven sites were only 12 and 9 percent of the total. Frequencies and percentages presented for this category add up to the sub-total included in the qualitative nonresponse study. 
NOTE: Sample sizes are rounded to the nearest 10, except for inconsistent UAA addresses sampled for the observation component (rounded to the nearest 5 due to small sample sizes). Percentages are rounded to one decimal place but have not been changed to 


reflect sample size rounding. Details may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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B.2. Qualitative Interview Methodology 


The purpose of the qualitative interviews was to gain a deeper understanding about how the 
participants inhabit their world and how that orientation may influence their thoughts on 
research and survey participation. 


B.2.1 Interview Recruitment 


As noted above, 400 addresses were sampled for the interview component of the study. As 
shown in exhibit B.2, several recruitment contact attempts were made. The interview 
recruitment materials can be found in appendix G. 


Recruitment began with an invitation letter that was mailed to all sampled addresses on 
April 19, 2019. The letter provided an overview of the study. It also asked the household 
members to contact the study team to schedule an in-person interview using the toll-free 
phone number or email address provided in the letter. The mailing included a $5 cash 
incentive, and the letter noted that the sample members would receive a $120 cash incentive 
for completing an interview. 


Exhibit B.2. Interview recruitment contact attempts 


Contact attempt Date 


Invitation letter April 19, 2019 
Reminder postcard 1 April 26, 2019 

ici 1 reminder calls to addresses that had a phone number on the sampling April 26 - May 2, 2019 
Week 1 in-person recruitment May 2 - May 9, 2019 
Reminder postcard 2 May 7, 2019 

Week 2 in-person recruitment May 15 - May 22, 2019 


Wave 2 reminder calls to addresses that had a phone number on the sampling 


frame May 22 - May 29, 2019 


Week 3 in-person recruitment May 29 - June 5, 2019 


In general, nonresponse follow-up continued until (1) the household scheduled an interview, 
(2) the household provided a refusal to participate (a single hard refusal or two soft refusals), 
or (3) field staff determined the address did not exist or was vacant. 


Mailed reminders. Two reminder postcards were mailed to sample members. Both 
provided an overview of the study, included contact information for setting up an interview, 
noted the $120 promised incentive, and mentioned that the study team would be in the 
neighborhood in the coming weeks. 


Phone reminders. Two waves of reminder calls were made to addresses that had a phone 
number on the frame (about 270 addresses). During each of the two waves, staff aimed to 
call each address up to two times. Some addresses were called less often (e.g., if the 
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household had already scheduled an interview before the second wave of calls or if initial 
calls indicated the available phone number was not associated with the sampled address) or 
more often (e.g., if the household requested a callback at a later time). To maximize the 
likelihood of reaching someone at home, calls were made on several days and at varied times 
throughout the day. During these calls, staff introduced the study, mentioned the promised 
incentive, and invited sample members to participate. Those who were interested were 
screened for eligibility and, if eligible, an interview was scheduled. For cases where the calls 
went unanswered, one voicemail was left per wave of calling. The voicemail introduced the 
study and encouraged sample members to contact the study team to schedule an interview. 


In-person visits: Three weeks of in-person recruitment were conducted. Two field staff 
teams (each comprised of a senior interviewer and a supporting field staff member) were 
assigned to visit half of the addresses in each site. Throughout the course of the study, 10 
senior interviewers and 11 field staff conducted recruitment efforts. Recruitment was 
conducted throughout the week, between 9:00 AM and 7:00 PM local time, but was primarily 
concentrated in the evenings and weekends. Teams attempted to visit each address at least 
once per week; depending on the outcomes of prior contact attempts, some addresses were 
visited more or less often than this. 


When no one answered the door, the team left a Sorry We Missed You card that encouraged 
the address to contact the study team to schedule an interview. When someone answered 
the door, the team introduced the study, mentioned the promised incentive, and invited the 
person who had answered the door to participate. The team addressed any questions or 
concerns raised by the household member, and a Commonly Asked Questions document was 
available for individuals who wanted more detailed information about the study. When 
interviews were scheduled for a later date, the team left an Appointment Reminder card and 
then contacted the interview participant shortly before the interview to confirm that the 
person was still available to participate. 


Case management and quality control. The outcomes of all call attempts and in-person 
visits were recorded by field staff in a case management system (see appendix E for more 
information about these outcomes). These outcomes were reviewed daily by site managers, 
who then provided feedback and guidance about next steps for recruitment. Site managers 
also conducted an oversight trip to each site to observe recruitment efforts and provide 
feedback to the teams. In addition, the qualitative team lead also held daily check-in meetings 
with the teams to discuss recruitment and interviewing outcomes. 


Recruitment language. Recruitment was conducted in both English and Spanish. Likely- 
Spanish-speaking households were sent recruitment bilingual materials that included both 
English and Spanish, were called by bilingual staff members, and received in-person visits 
from bilingual field staff. When Spanish language barriers were encountered in the field (e.g., 
an English-only staff member called or visited a Spanish-speaking household), the case 
received a return call or return visit by a Spanish-speaking staff member whenever possible. 


Eligibility criteria. Individuals who were interested in completing an interview were 
screened for eligibility. To be eligible to participate in the interview, individuals had to: (1) 
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be age 18 or older, (2) live or stay at the sampled address most of the time between January 
and April 2019 (the time period during which NHES:2019 screener phase mailings were 
sent), and (3) and have some responsibility for handling the household’s mail. Within- 
household sampling was not conducted. 


B.2.2 Interview Protocol 


Eighty-five interviews were completed between May 3 and June 5, 2019. The interviews 
were conducted using semistructured, conversational methodology. This approach 
facilitated exploration into both hypothesized drivers of nonresponse and areas of 
importance to the participants that were not already captured by available variables, such 
as those on the sampling frame (e.g., estimated household income, route type) or that are 
collected as part of the survey (e.g., notification from the postal service that a mailing was 
unable to be delivered as addressed). 


The interview team comprised a senior interviewer who led the conversation and a field staff 
who took notes. Interviews generally took place at the sampled address. Per participants’ 
request, a few interviews took place at a quiet public place, such as the apartment building 
lobby or a coffee shop. Interviews were conducted in English and Spanish and lasted 
approximately 90 minutes. In most cases, a single household member completed the 
interview. However, in about five interviews, a second household member participated as 
well. All interviews began with the participant reading an informed consent form and the 
senior interviewer answering any questions that arose. 


Interview domains. Interviewers were trained to cover all of the domains included in the 
interview protocol during the interview and to probe other topics when raised by the 
participant that may be germane to the overarching research questions. Interviewers were 
free to address the domains in whatever order felt comfortable, conversational, and natural. 
The protocol included example questions for each domain, but there were not any required 


questions (see appendix G for the interview protocol and other materials used during the 
interview). The domains included: 


e Household make-up 

e Experience with mail delivery 

e Understanding of and attitudes toward surveys 
e Privacy concerns 

e Attitudes toward the government 

e Education 

e Time use 


e Civic and community engagement 
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The interviews also contained two slightly more structured activities: (1) the example mail 
activity and (2) the NHES:2019 screener materials review activity. 


Example mail activity. In the first activity, the interviewer handed the participant an 
example mail bundle that contained 12 mailings typically found in a routine mail delivery 
(see appendix G for the example mailings). One of the mailings was the NHES:2019 initial 
screener package. Mailing features (such as color, font, return and sender addresses, and 
postage type) were varied to assess whether—and how—they influenced the participant’s 
interest in engaging with the mailings. All participants received the same mailing pieces, but 
the pieces were bundled together in different orders across participants. All mailings were 
addressed to a fictional address: 123 Main Street, AnyTown, ST 54321. They were also 
addressed to a fictional person: Jamie Smith (for some mailings, the recipient name was 
instead listed as the Smith household, member of AnyTown household, or current resident). 
Participants were asked to imagine that they were that person and lived at that address. 
Participants were then asked to sort the mail bundle according to their usual habits and 
explain how they would react to each mailing and why: Would they open it? Would they read 
it? Would they keep it? All mailings were sealed, and participants’ reactions were limited to 
the exterior features of the mailings. 


NHES:2019 screener materials review activity. In the second activity, the interviewer 
showed the participant five NHES:2019 screener mailings. Participants opened and 
reviewed the mailings. The interviewers asked if participants remembered receiving such a 
mailing, if so, and what they had done with it after receiving it. The interviewers also asked 
participants for feedback about the mailings and the paper screener (see appendix F for the 
NHES:2019 screener materials). 


At the conclusion of the interview, the interviewer read aloud a demographics form that 
contained questions about the participant and the household. A show card was used for the 
household income question, which allowed participants to point to an income range on the 
card. Leading up to and during the interview, when possible, the team observed 
unobtrusively the household exterior and interior. At the end of the interview, the team then 
gave the participant the $120 cash incentive. 


Notetaking and memos. Interviews were recorded with participants’ permission, with the 
field staff's notes serving as a backup source of information. However, about 10 participants 
declined to be recorded; in these cases, the field staff's notes served as the record. After each 
interview, the interviewer wrote a memo summarizing the key discussion points and themes 
from both recruitment and the interview, any observations about the participant that would 
not be captured on the audio recording (e.g., body language, facial expressions), and a 
description of the interior and exterior of the home. 


Training, monitoring, and quality control. All interviewers attended a two-day, in-person 
training that provided background on the NHES and the qualitative nonresponse study, 
covered recruitment and interviewing protocols, and included opportunities to practice 
recruitment and interview skills. All field staff completed a two-hour virtual training that 
provided background on the NHES and the qualitative nonresponse study and reviewed the 
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responsibilities relevant to this role (e.g., notetaking, navigation, utilizing the case 
management system); because most field staff were also observers, they also completed a 
five-hour virtual training as part of that role. 


Interviewers were observed in the field by site managers, and recordings were monitored 
throughout the field period to ensure the interviews were being conducted accurately and 
respectfully and interviewers were obtaining the needed information. 


Short interview. Once two soft refusals had been received from an address, the team asked 
if the household member would instead be willing to answer a short set of questions. After 
screening for eligibility, the short form questions asked whether the participant recalled 
receiving any mail from the Department of Commerce or the Census Bureau, whether the 
participant recalled receiving any mail about the National Household Education Survey, and, 
if so, the participant’s reasons for not responding to the survey. Six short form interviews 
were completed. 


B.3. Address Observation Methodology 


The purpose of the address observations was to record the characteristics of nonrespondent 
and inconsistent-UAA addresses in a systematic manner to (1) help understand reasons for 
nonresponse not already captured by available variables and (2) evaluate the accuracy of 
sampling frame data. 


B.3.1 Observation Fieldwork 


As noted above, 760 addresses were sampled for the address observation component of the 
study. Most observations were conducted over a four-day period (April 29 through May 2, 
2019) between the hours of 8:00 AM and 5:00 PM local time.® Seventeen staff members 
conducted observations; most observers working in qualitative interview sites also served 
as field staff for the interview component of the study. Each observer was assigned 40 to 50 
addresses to observe, depending on the site at which they were working. 


Observers drove to each of their assigned addresses. After arriving at an address, they drove 
or walked around the area to get a sense of the overall neighborhood but conducted the 
observation from the car as best as they could. They completed the observation from public 
property, and, whenever possible, facing the front of the address. For apartment buildings, 
they also attempted to gain access to the building via a manager or security guard in order 
to observe a mail room or area and the door or balcony of the sample unit. They completed 
the observation from public property, and, whenever possible, facing the front of the 
address. 


When possible, observers began by discretely taking two or three photos of the exterior of 
each address, which were used in a fieldwork tracking effort. They next used a tablet to 


6 A small number of observations were conducted after May 2 or outside of the core observation hours. This was typically because of 
logistical issues (e.g., an address being far away from all of the other sampled addresses or challenges accessing the area where the 
address was located). 
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complete a web-based observation instrument, which had been programmed using the IES 
Survey Engine. In a small number of cases, internet was not available while completing the 
observation; these observations were recorded on paper forms and then entered into the 
web instrument as soon as the observer had an internet connection. When completing 
observations of addresses that were part of multi-unit buildings, observers were instructed 
to limit their observation only to those items that could clearly be attributed to the sampled 
unit (e.g., an infant swing on the unit’s balcony would be considered evidence of the presence 
of children, but a playground on the grounds of the larger apartment complex would not). 
Most addresses were only visited once, but in a small number of cases, the observer had to 
return to the address a second time to complete the observation (e.g., due to a road being 
closed or blocked on the first visit). On average, each observation took about 7 to 10 minutes 
to complete. 


Training. All observers completed a five-hour virtual training that provided background on 
the NHES and the qualitative nonresponse study, a detailed review of an observation 
codebook, and a summary of the responsibilities relevant to this role (e.g., navigation, 
completing observations, utilizing the case management system). 


Case management and quality control. The outcome of each observation attempt was 
recorded in a case management system. For those addresses sampled for the interview 
component of the study, observers also recorded notes about the appearance of the address 
to facilitate returning to the address for interview recruitment. Observation outcomes were 
reviewed daily by site managers, who then provided feedback and guidance about next steps 
for ensuring all of the observations were completed during the field period. Oversight trips 
were conducted in a few of the sites to monitor observation efforts and provide feedback to 
the observers. 


B.3.2 Observation Instrument 


Observers were asked to use their best judgement when completing the observation 
instrument and to avoid any blind guesses or assumptions. To encourage this, “could not 
determine” and “other” response options were provided for many items, as well as a text box 
for providing more detailed information about what led them select their response to that 
item. Observers were instructed to be extremely thorough in their observation and include 
rich descriptive information in all write-in fields, especially when they selected a “could not 
determine” or “other” category. 


The observation instrument began with a question asking if the sampled address was able 
to be observed. Observers noted that the address was either fully observed, partially 
observed, or not able to be observed. Partially observed addresses are those where the 
observer was able to observe the exterior of the multi-unit building but was not able to enter 
the building to observe the interior entry to the sampled unit. When an address was not able 
to be observed, the observer indicated why this was the case (e.g., could not locate, could not 
get close enough to confirm whether address exists). As shown in exhibit B.3, based on this 
preliminary observation outcome, observers then responded to up to 10 additional forced- 
choice items that captured two types of information about sampled addresses: physical 
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characteristics and household characteristics. The full observation instrument can be found 
in appendix H. 


Exhibit B.3. Observation items 


Item Characteristic type Addresses for which this item was captured 

Observability N/A All addresses 

Structure type Physical Full or partially observed addresses 

Occupancy status Physical Fully or partially observed residential addresses 

Mail access type Physical Fully or partially observed residential addresses that 
are not permanently vacant 

Presence of children Household Fully observed, occupied residential addresses 

Privacy or security Household Fully observed, occupied residential addresses 

concerns 

Pride in education Household Fully observed, occupied residential addresses 

Patriotism Household Fully observed, occupied residential addresses 

Community involvement Household Fully observed, occupied residential addresses 

Other household Household Fully observed, occupied residential addresses 

attributes 

Household income Household All addresses that are not vacant lots or commercial 
addresses 


Physical characteristics. The first few items captured information about the physical 
characteristics of the address, including structure type (single-unit, duplex, apartment, 
commercial, etc.) occupancy status (occupied, permanently vacant, etc.), and mail access 
type (mail slot, mailbox at the end of the driveway, etc.). 


Household characteristics. The remaining items captured information about the 
characteristics of the household members. First, observers indicated whether there was 
evidence that there were children living in the household (e.g., children’s bikes or toys)—or 
if there was evidence that it was unlikely that children lived there (e.g., address located in 
retirement home, AARP bumper sticker). Next observers answered a series of questions 
asking whether there was evidence of any of the following household characteristics: having 
pride in education, valuing community involvement, valuing patriotism, being concerned 
about privacy or security, and speaking a language other than English. There was also a place 
for observers to note evidence of any other household interests or values that they felt were 
important to record. Finally, observers indicated whether the household’s income appeared 
to be in the bottom third, middle third, or top third of the U.S. population. 


B.4 Data Processing 


After the data collection was completed, several steps were taken to clean the interview and 
observation data and prepare them for analysis. 
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B.4.1 Interview Data Processing 


Interview data processing proceeded in several steps. First, transcripts of the interview 
recordings were prepared. Next, these transcriptions were coded in NVivo. 


Transcription. The interview recordings were sent to a qualified professional transcription 
firm. Once transcribed, the transcripts were returned to AIR via a secure server. AIR 
conducted a quality review of all transcripts to assess transcriber accuracy and relistened to 
portions of select audio files to correct, when possible, inaudible sections of the transcript. 
For interviews where the participant declined to be recorded, the notes taken by the field 
staff were used in place of an interview transcript. All Spanish-language interview 
transcriptions were translated to English for analysis. All PII was removed and replaced with 
bracketed text that allows the reader to still understand the text’s meaning (e.g., [apartment 
number]). All transcripts (and notes when transcripts were not available) were 
subsequently uploaded to NVivo. This collection of transcripts and notes was the starting 
point for the qualitative analysis. 


Coding. Once the interview transcripts were uploaded to NVivo, a series of codes were 
applied to segments of text in the transcripts. These codes captured participants’ 
experiences, opinions about who they are, how they spend their time, and why the 
household did not respond to NHES:2019. They were selected in two stages. In the first 
stage, the lead analysts reviewed the interview protocol, which included interview domains 
and sample questions. Given that senior interviewers were instructed to cover all domains 
in everyinterview, the codebook started with a _ set of deductive codes 
that corresponded with each domain - such as privacy, surveys, and time use. In the second 
stage, the lead analysts reviewed theinterviewer memos. Through reading these 
descriptions of discussions with participants, as well as the observations and reflections 
from senior interviewers and field staff, the team refined the deductive codes and added 
inductive codes, such as “Life is hard.” The final codebook contained the 15 main codes 
shown in exhibit B.4. 


Each main code contained4 to 20 subcodesthatcaptured more detailed or 
nuanced information about the broader topic (e.g., “daily routine” subcode within “Time 
Use”). When applicable, coders aimed to primarily use the more detailed subcodes; however, 
if the discussion did not fit under a specific subcode, yet addressed the concepts conveyed in 
the main code, the relevant main code was applied. Whenever a subcode was applied, the 
“parent” main code was automatically applied as well. The codebook also had “adjective 
codes,” which were only used in conjunction with main codes and subcodes to add detail, 
depth and context about the discussion (e.g., “would not read” adjective code could be used 
with the “Initial screener mailing” subcode). The full codebook can be found in appendix G. 
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Exhibit B.4. Main codes included in qualitative interview codebook 


Code Description 


Who’s who Any discussion that mentions the people the participant or household members interact with, now or in 
the past. 

Time use General discussion about how participants or household members spend their time daily. 

Government and General discussion about the business of operating the United States, states, or local municipalities, 

politics including the opinions of participants, household members, or other people important to the 
participants about political parties or domestic and international policies. 

Education General discussion about the opinions of and experiences with education concerning the participant, 
household, or people important to them as well as thoughts about education in general. 

Surveys General discussion about surveys and research, as well as opinions about and reactions to particular 
surveys (excluding the NHES, which had its own code). 

Technology General discussion about technology and how participants, household members, or people important to 
them use it. 

Privacy General discussion about the opinions and experiences with privacy by the participants, household 
members, or people important to them. 

Community General discussion about how the participant, household members or other people important 
to them interact with and perceive the environment outside the household. 

Life is hard General discussion about barriers and obstacles participants, household members, or people important 
to them face or have faced in life and their reactions to them. 

Beliefs General discussion about what participants, household members, or people important to them believe 
or value, as well as the underlying principles that influence how they prioritize their time. 

Identity General discussion about demographics that are salient to participants, household members, or people 
important to them. Discussion does not have to be about the participants’ personal identity. 

Mail delivery General discussion about participants’ and households’ experience with receiving mail and sending 
mail. 

Example mail Discussion about the example mail activity including participants’ reactions to all aspects of 


the mailings and reasons why they would or would not open it. 


NHES Any discussion related to the National Household Education Survey, the survey materials, or 
the qualitative nonresponse study. 


Memos Any information from the interviewer memo or field staff notes that provides information not 
otherwise captured in the transcript. 


Prior to coding, a team of eight coders attended two trainings that provided an overview of 
the study’s aims, the codes and their definitions, and inclusion and exclusion criteria for 
applying each code. Coders also participated in an interactive session where they learned 
how to apply the codes to text segments within the transcript using NVivo. During the 
training coders were taught to spot multiple topics within a single sentence or paragraph 
and to apply all the relevant codes accordingly. 


The coders began by coding 10 percent of the transcripts, which were then reviewed by a 
lead analyst for accuracy and quality. Based on the initial round of coding, additional 
subcodes were added to capture ideas that emerged from the data that were not represented 
by the existing codes. Transcripts that had already been coded were revisited to add the 
newly created codes. The team then coded the remainder of the interviews, all of which were 
reviewed by a lead analyst for accuracy and quality. 


The team conducted a similar process for creating attribute variables, which are discrete 
variables in NVivo that further describe the participant demographics or behavior. A list of 
deductive attributes was created using the demographic forms completed during the 
interview and appended to the appropriate transcript. Through the analysis process, the 
team created inductive attributes such as mail processing behavior, recall of NHES mailings, 


B-22 


Appendix B. Additional Qualitative Nonresponse Study Methodology 


and nonresponse typology category. Once created, these attributes were appended to the 
transcripts as well. 


Analysis. Codes are an efficient way to organize massive amounts of textual data and help 
the team conduct analyses but cannot themselves answer research questions. Therefore, 
after all transcripts were coded, a team of five analysts analyzed the coded transcript data 
via multiple readings of the text. Most of the analysts had participated in earlier parts of the 
study (e.g., as interviewers), but any new team members received an onboarding training. 


The analysts first read and summarized each code and subcode to synthesize what 
participants said about a given topic, as well as noting where responses seemed thematically 
similar or diverged. Once they outlined the overall findings for each code and subcode, they 
used the matrix coding feature in NVivo to compare the responses among participants with 
different demographic characteristics. This approach allowed the research team to elucidate 
how deductive groups (such as race or ethnicity) or inductive groups (such as mail sorting 
behavior) did or did not vary when looking at experiences, attitudes, and behaviors. A lead 
analyst regularly met with the team members, reviewed their work, and provided feedback. 


B.4.2 Observation Data Processing 


Several steps were taken to prepare the observation data for analysis, including removing 
duplicate entries, reviewing and coding write-in responses and removing PII. 


Initial cleaning. Data processing began with a series of initial cleaning steps, such as 
reviewing study IDs (which had been hand-entered into the observation instrument) for 
accuracy and resolving any duplicated entries for individual cases (e.g., on rare occasions, 
the observer lost internet connectivity during the observation and had to complete the 
observation form in two parts). 


The observation instrument included several items with skip patterns. However, due to 
limitations in the online instrument software (e.g., skips could only be programmed based 
on items on the previous webpage), it was not possible to program all the desired skip 
patterns for three of the items. These skips were corrected during data processing to ensure 
that all valid skips were accounted for and marked appropriately. 


Review and coding of write-in responses. The observation instrument also contained 
eight text fields. These text fields were manually inspected to remove PII. They were also 
reviewed and coded in the ways described below. 


e Some of the write-in items asked staff to explain a specific substantive answer to a 
forced-choice item (e.g., description of the evidence leading the observer to conclude 
the household members had privacy concerns or were involved in their community). 
The responses to these write-in items were reviewed to confirm that the correct 
response had been selected for the forced-choice item. Responses to the forced- 
choice items were recoded where necessary. 
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e Other write-in items asked staff to provide more information about why they had 
selected an “other” or “could not determine” response to a forced-choice item. These 
write-ins were reviewed to determine if any of the responses could be upcoded into 
existing categories. They were also reviewed to determine if any new categories 
should be created (e.g., an additional mail access type or structure type); however, 
ultimately, no new categories needed to be created. 


e Finally, a few of the write-in items simply provided the opportunity for observers to 
enter additional rich description of the address that were not captured elsewhere: (1) 
the item that asked observers to provide a general description of the address and (2) 
the item that asked observers if there was evidence of any other household attributes 
of note. These responses were reviewed to determine if information provided for 
these items could be upcoded into existing items (e.g., if an observer noted evidence 
of patriotism in these items instead of in the patriotism item). The responses to these 
items were also reviewed by a senior team member and, based on themes identified 
in the responses, a set of new codes was developed and applied to the responses. 
These codes included: importance of religion, internet or television connectivity, 
welcoming decor, outdoor living, and other outdoor decor. 


In addition to the approaches outlined above, a targeted word search was also conducted for 
several variables, particularly the newly created ones, to determine whether these 
characteristics were mentioned in write-ins other than the one originally associated with 
that characteristic. 


A team of four coders reviewed the write-in responses, applied codes, and flagged forced- 
choice items in need of recoding. After completing a training session, each coder was 
assigned approximately 200 cases; about 10 percent of the cases were double coded. The 
coders started by reviewing and coding a subset of the write-in responses; these were 
reviewed by a senior team member for accuracy and feedback was provided to the coders. 
The coders then reviewed and coded to the remainder of the write-in responses. Two senior 
team members reviewed the codes for accuracy. 
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Before we began the observation analyses, we conducted two preliminary sensitivity 
analyses to determine whether certain types of addresses should be included in the 
observation results discussed throughout this report: (1) addresses that responded after the 
fourth NHES screener package (“late respondents”); and (2) addresses that had only the 
fourth NHES screener package returned as undeliverable (UAA). This appendix presents the 
results of those analyses. 


C.1 Late NHES Respondents 


To be eligible to be sampled for the qualitative nonresponse study, addresses had to have 
not responded to NHES:2019 as of the date the fourth screener package was sent (April 11, 
2019). As a result, it remained possible that some of the addresses sampled as 
nonrespondents would end up responding after the fourth screener package. This ended up 
being the case for about 80 of the 700 addresses that were sampled as nonrespondents. 


Key takeaway: Late respondents differed from final nonrespondents on several variables of 
interest. Many of these differences were consistent with prior literature in terms of 
characteristics that are associated with a higher likelihood of response to the NHES—and to 
surveys more generally (e.g., older, higher income). As a result, and due to the large number 
of notable differences, we determined that late respondent addresses should be excluded 
from the nonrespondent observation analyses conducted as part of this report.” 


Methods: As shown in tables C.1 and C.2 at the end of this section, we compared the 
characteristics of the late respondents to those of the addresses that never responded in 
terms of both information available on the NHES sampling frame and the observations 
collected as part of the qualitative nonresponse study. The purpose of this analysis was to 
determine if late respondents were different enough from those that never responded to 
exclude them from the analyses of “nonrespondent addresses.” 


Results: We found that late respondents differed from nonrespondent addresses for several 
of the analyzed characteristics. For each of the characteristics listed below, we found a 
significant bivariate relationship between the overall characteristic and late respondent 
status. We then summarized patterns in terms of which characteristics were more common 
or less common among late respondents than among final nonrespondents. In general, late 
respondents were less likely to be missing information for household-level sampling frame 
variables; they also were more likely to be fully observed during the observation component 
of the qualitative nonresponse study. In addition, they more closely resembled the 
characteristics of groups that tend to be more likely to respond to the NHES—and to surveys 
more generally (e.g., older, higher household income). Additional detailed findings are 
included below. 


e Observation outcome: Late respondents were more likely to be fully observed and 
less likely not to be located. 


7 Late respondents were, however, included in the interview analyses, and “late respondent” status was used as a subgroup variable 
when interpreting the interview findings. 
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Structure type: Late respondents were more likely to be observed to be single-unit 
structures and less likely to be observed to be apartments. The same pattern was seen 
for the route type and dwelling type variables available on the NHES sampling frame. 


Household income: Late respondents were more likely to be observed to have 
household incomes in the top third of the population and less likely to be observed to 
have household incomes in the bottom third. The same pattern was seen for the 
household income variable available on the NHES sampling frame. 


Observed household attributes: Late respondents were more likely to be observed 
to have indicators of outdoor living and the importance of religion. 


Bilingual mailing status: Late respondents were more likely to have been sent 
English-only materials for all NHES:2019 mailings. 


Age of head of household: Late respondents were more likely to have a head of 
household who was age 65 or older and less likely to be missing this information on 
the frame. 


Presence of children: Late respondents were more likely to be flagged on the 
sampling frame as having children living in the household. This pattern was not 
repeated for the presence of children observation. 


Number of adults: Late respondents were more likely to have 2 adults in the 
household. 


Phone number available: Late respondents were more likely to have a phone 
number available on the sampling frame. 


Home tenure: Late respondents were more likely to own their residence and less 
likely to rent it. 


There were also significant differences between late respondents and final nonrespondents 
in terms of the area-level characteristics available on the sampling frame, but the patterns to 
these differences were not always clear. A few of the more notable differences include the 
following: 


Late respondents were more likely to live in blocks with the lowest percentage of 
persons without a high school diploma. 


Late respondents were less likely to live in blocks with the highest Low Response 
Scores (that is, with the lowest expected mail return rates to the Decennial Census). 


Late respondents were more likely than final nonrespondents to live in areas with 
higher rates of high-speed internet connectivity. 
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Table C.1. Percentage distribution of selected observed characteristics, by final 
NHES:2019 screener response status: 2019 


Final NHES:2019 screener 
response status! 


Number of Late Final Chi-square 
Selected observed characteristics addresses respondent _nonrespondent statistic 
Total 700 80 620 Tt 
Observation outcome 10.64 * 
Observed 530 79.8 74.8 
Partially observed, could observe the exterior of the 
multi-unit building but not the sampled unit? 20 $ 3.1 
Not observed, could not locate address 120 10.7 17.4 
Not observed, could not access address 20 t 2.6 
Not observed, other reason 20 7.1 2.1 
Structure type? 13.20 * 
Single-unit 320 61.8 48.3 
Duplex 40 5.3 6.0 
Townhouse or rowhouse 50 11.8 6.5 
Low-rise apartment* 160 11.8 25.9 
Mid-rise apartment* 40 2.6 74 
High-rise apartment* 30 5.3 5.1 
Could not determine or missing 10 t 0.7 
Residential occupancy status® 4.50 
Occupied residential unit 470 82.9 71.5 
Residential unit, could not determine occupancy status 160 15.8 26.6 
Vacant residential unit® 10 $ 1.8 
Vacant lot t # $ 
Commercial business # # # 
Mail access type” 12.55 
Mail slot 30 27 5.7 
Mailbox attached to the home 190 22.7 30.7 
Mailbox at the end of the driveway 140 36.0 19.2 
Mailbox across the street or at the end of the road 70 10.7 11.2 
Mailbox, slot, or room in multi-unit building 90 10.7 14.5 
More than one type of mail access® # # $ 
No mailbox or slot in view 50 6.7 7.8 
Could not determine or missing 70 10.7 10.6 
Household income 13.55 * 
Bottom third of population 210 21.4 314 
Middle third of population 340 50.0 48.0 
Top third of population 100 25.0 12.2 
Could not determine 60 3.6 8.5 
Privacy or security concern?.1° 2.78 
Yes 170 40.9 30.7 
No 350 59.1 69.3 


See notes at end of table. 
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Percentage distribution of selected observed characteristics, by final 


NHES:2019 screener response status: 2019—Continued 


Final NHES:2019 screener 
response status! 


Number of Late Final Chi-square 
Selected observed characteristics addresses respondent _nonrespondent statistic 
Outdoor living? 7.16* 
Yes 140 31.0 18.5 
No 560 69.0 81.5 
Presence of children?12 0.01 
Yes 110 21.2 218 
No 400 78.8 78.2 
Patriotism? 13 0.00 
Yes 40 7.6 78 
No 480 92.4 92.2 
Welcoming decor?14 0.18 
Yes 40 6.0 4.9 
No 670 94.0 95.1 
Internet or television connectivity?.15 0.08 
Yes 40 6.0 5.2 
No 660 94.0 94.8 
Speaking a language other than English?1¢ 1.07 
Yes 30 3.0 6.2 
No 490 97.0 93.8 
Community involvement?17 0.88 
Yes 20 £ 3.8 
No 500 98.5 96.2 
Importance of religion?.18 8.78 * 
Yes 10 6.0 1.3 
No 690 94.0 98.7 
Pride in education®19 0.01 
Yes 10 £ 1.3 
No 510 98.5 98.7 
Other outdoor decor?2° 1.73 
Yes 130 23.8 17.9 
No 570 76.2 82.1 
tNot applicable. 


#Rounds to zero. 


+Reporting standards not met. There are too few cases for a reliable estimate. 


*p< 0.05. 


1Late NHES respondents are those that responded after the fourth screener package was sent. Final nonrespondents are those that did not end up 


responding to NHES:2019. 


2Partially observed addresses are those w 


to observe the interior entry to the sampled unit. 


3Structure type observations were not col 
nonresidential. 


ected for addresses that could not be observed (e.g., cannot locate the add 


here the observer was able to observe the exterior of the multi-unit buildin, 


g but was not able to enter the building 


ress) or were observed to be 


4Low-rise apartment buildings are those with 1 to 3 floors. Mid-rise apartment buildings are those with 4 to 6 floors. High-rise apartment buildings are those 


with 7 or more floors. 
5Residential occupancy status was not col 
6Vacant residential unit includes (1) temp: 


ected for addresses that were not able to be observed (for example, for ad 
orarily vacant addresses where the unit had a for sale or for rent sign or a 


current occupants, (2) seasonally vacant addresses where the unit was located in a resort area and was well-mainta 


tenants during other parts of the year, bu 


uninhabitable (e.g., boarded windows/doors, holes in the walls or roof), was condemned or was under construction. 


there were no evidence of current occupants, and (3) permanently vacan 


dresses that could not be located). 

lock box and there was no evidence of 
ined enough to suggest that it had 
addresses where the address appeared 


7Mail access type observations were not collected for addresses that could not be observed (e.g., cannot locate the address) or were observed to be 


nonresidential or permanently vacant. 


8This category includes addresses where mail could have been received in more than way (e.g., a house that has bot 


the end of the driveway). 


h a mail slot in the door and a mailbox at 
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*Household attributes were not collected for addresses that could not be observed (e.g., cannot locate the address) or for which only a partial address 
observation could be completed (e.g., can observe the multi-unit building but cannot gain entry to observe the sampled unit). Household attributes also were 
not collected for addresses that were observed to be nonresidential or temporarily or permanently vacant. 

10Addresses with privacy or security concerns are those where observers identified indicators that privacy or security is important to those living in the 
sampled unit. Examples include surveillance cameras, driveway gates, security company signs or stickers, and "No trespassing" signs. For multi-unit 
buildings, indicators that focused on the larger building in which the unit was located, such as entry buzzers or fences, were not included because they could 
be not directly associated with the sampled unit. 
11Addresses that value outdoor living are those where observers identified indicators that people in the sampled unit spend time outdoors. Examples include 
patio furniture, porch swings/benches, swing sets, sporting goods, and grills. This variable was added during data processing based on patterns observed in 
write-in responses. 
12Addresses where children are thought to be present are those where observers identified indicators that suggest children live in the sampled unit. 
Examples include toys, bikes, car seats, strollers, outdoor swings/play sets, and child finder stickers for firefighters. 

13Patriotic addresses are those where observers identified indicators of American national or state-specific patriotism. They also include those with 
indicators of current or past involvement with the U.S military or pride in the U.S. military. 
14Addresses with welcoming decor are those where observers noted the presence of an object that welcomes visitors to the sampled unit, such as a welcome 
mat or welcome sign that explicitly says “welcome.” This variable was added during data processing based on patterns observed in write-in responses. 
15Addresses with internet or television connectivity are those where observers noted the presence of a satellite, cable, DirectTV dish or other electronic 
equipment, or a sign that indicates television or internet connectivity at the sampled unit. This variable was added during data processing based on patterns 
observed in write-in responses. 

16Addresses that speak a language other than English are those where observers identified indicators that people in the sampled unit speak a language other 
than English. Examples include flags for non-English-speaking countries, in-home business signs, yard/window signs, or bumper stickers written in another 
language at the sampled unit; or a large number of flags, signs, or bumper stickers in (or including) another language in the surrounding neighborhood (even 
if indicators are not observed at the sampled unit). 

17Addresses with community involvement are those where observers identified indicators that people in the sampled unit are involved in the community. 
Examples include political candidate signs or indicators of being involved with charities, kids’ sports/clubs, or neighborhood associations. 

18Addresses that value religion are those where observers identified indicators that suggest that religion is important to the people in the sampled unit. 
Examples include “Bless this House” signs, religious figures or statues, or Mezuzahs. This variable was added during data processing based on patterns 
observed in write-in responses. 
19Addresses with pride in education are those where observers identified indicators that education is important to the sampled unit. Examples include 
school, college and/or university flags or stickers; honor roll bumper stickers; or other indicators of pride in a child’s school. 

20Addresses with other outdoor decor are those where observers noted the presence of items used to decorate the exterior of the sampled unit. Examples 
include gnomes, flamingos, yard art, decorative flags, wreaths, Easter decorations, water features, or windchimes. This attribute focuses only on decorative 
items and does not include items that represent another household attribute shown in the table (e.g., American flags were coded under patriotism only). This 
variable was added during data processing based on patterns observed in write-in responses. 

NOTE: Addresses that were sampled for the qualitative nonresponse study in the "inconsistent UAA outcomes" group were excluded from this analysis. 
Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but have not been changed to reflect sample size rounding. Detail 
may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table C.2. Percentage distribution of selected observed characteristics, by final 
NHES:2019 screener response status: 2019 


Final NHES:2019 screener 
response status! 


Number of 
Selected characteristics addresses 
Total 700 
NHES:2019 screener mailings language 
All bilingual mailings (English and Spanish) 280 
Mix of bilingual mailings and English-only mailings 340 
All English-only mailings 90 
Age of head of household2 
18-34 60 
35-44 100 
45-54 110 
55-64 90 
65 and older 90 
Missing 250 
Gender of head of household? 
Male 220 
Female 200 
Missing 280 
Education of head of household? 
Less than high school credential 150 
High school credential 110 
Some college 140 
Bachelor's degree 90 
Graduate degree 50 
Missing 160 
Race of head of household? 
White, non-Hispanic 230 
Black, non-Hispanic 110 
Hispanic 160 
Other race, non-Hispanic 40 
Missing 160 
Household income2 
Less than $50,000 280 
$50,000-$74,999 80 
$75,000-$99,999 80 
$100,000 or higher 170 
Missing 90 
Household flagged as having children2 
Yes 180 
No 520 


See notes at end of table. 


Late 
respondent 


80 


44.0 
36.9 
19.0 


8.3 
15.5 
17.9 
11.9 
25.0 
21.4 


33.3 
27.4 
39.3 


15.5 
22.6 
15.5 
17.9 
11.9 
16.7 


46.4 
11.9 
21.4 

2.4 
17.9 


27.4 
13.1 
22.6 
26.2 
10.7 


39.3 
60.7 


Final 
nonrespondent 


620 


39.3 
49.5 
11.2 


8.9 
14.1 
15.6 
13.3 
10.9 
37.2 


30.8 
28.9 
40.3 


22.4 
14.8 
20.5 
12.7 

6.8 
22.9 


31.2 
16.6 
22.6 

6.0 
23.7 


42.0 
11.5 
10.1 
23.2 
13.1 


23.2 
76.8 


Chi-square 
statistic 


tT 
6.61 * 


17.40 * 


0.22 


10.83 


9.10 


14.79 * 


10.14 * 


C-7 


Appendix C. Observation Sensitivity Analyses 


Table C.2. Percentage distribution of selected observed characteristics, by final 
NHES:2019 screener response status: 2019—Continued 


Number of 

Selected characteristics addresses 
Number of adults in household2 

1 adult 390 

2 adults 160 

3 adults or more 60 

Missing 90 
Phone number available2 

Yes 440 

No 260 
Route type? 

Street 440 

High rise 260 
Dwelling type? 

Single-unit 410 

Multi-unit 290 
Home tenure2 

Own 360 

Rent 220 

Missing 120 
Urbanicity2 

Urban 310 

Suburban 350 

Town 10 

Rural 30 
Region2 

Northeast 100 

South 300 

Midwest 200 

West 100 
Race/ethnicity stratum2 

25% or more Black 170 

40% or more Hispanic 190 

Other 330 
Tract poverty rate? 

Less than 20% 470 

20% or more 230 


Final NHES:2019 screener 
response status! 


Late 
respondent 


47.6 
35.7 

6.0 
10.7 


82.1 
17.9 


75.0 
25.0 


72.6 
27.4 


65.5 
21.4 
13.1 


38.1 
56.0 


+ 
4.8 


16.7 
46.4 
25.0 
11.9 


16.7 
19.0 
64.3 


76.2 
23.8 


Final 
nonrespondent 


56.5 
21.8 

8.6 
13.1 


60.6 
39.4 


60.6 
39.4 


56.3 
43.7 


49.0 
33.4 
17.5 


45.6 
48.9 
1.0 
4.5 


14.0 
42.4 
29.1 
14.6 


25.8 
28.7 
45.5 


66.2 
33.8 


Chi-square 
statistic 


8.16 * 


14.80 * 


6.57 * 


8.07 * 


8.11 * 


1.72 


1.47 


10.51 * 


3.33 


See notes at end of table. 
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Table C.2. Percentage distribution of selected observed characteristics, by final 
NHES:2019 screener response status: 2019—Continued 


Selected characteristics 


Percent of households in Census block that 
include a child? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Percent of persons in Census block that speak a 
language other than English? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Percent of persons in Census block without a high 
school diploma or the equivalent? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Low Response Score* 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Residential high-speed internet per 1000 
households> 
600 or less 
601-800 


801 or more 
+ Not applicable. 


+ Reporting standards not met. There are too few cases for a reliable estimate. 


* p< 0.05. 


Number of 
addresses 


190 
140 
180 
200 


60 
120 
220 
290 


180 
130 
150 
240 


140 
120 
160 
290 


160 
250 
290 


Final NHES:2019 screener 
response status! 


Late 


respondent 


21.4 
14.3 
35.7 
28.6 


10.7 
22.6 
41.7 
25.0 


35.7 
26.2 
22.6 
15.5 


31.0 
22.6 
26.2 
20.2 


13.1 
33.3 
53.6 


Final 
nonrespondent 


27.1 
20.1 
24.2 
28.6 


8.9 
16.4 
30.4 
44.3 


24.2 
18.2 
21.4 
36.2 


18.4 
16.3 
21.8 
43.6 


23.7 
36.4 
39.9 


Chi-square 
statistic 


6.04 * 


11.48 * 


15.78 * 


18.15 * 


7.20 * 


1Late NHES respondents are those that responded after the fourth screener mailing was sent. Final nonrespondents are those that never responded to 


NHES:2019. 


2These characteristics are based on variables available on the NHES:2019 sampling frame. 
3These characteristics are based on American Community Survey (2013-2017) five year estimates. Cases in the first quartile were those with the lowest 
prevalence of the characteristic in question and those that are in the fourth quartile are those with the highest prevalence. 
4The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. A higher score corresponds to a lower expected mail return rate. This variable was not available for a very small number of cases, and these cases 


have been excluded from this analysis. 


5This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 


NOTE: Addresses that were sampled for the qualitative nonresponse study in the "inconsistent UAA outcomes" group were excluded from this analysis. 
Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but have not been changed to reflect sample size rounding. Detail 


may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 


2017. 
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Appendix C. Observation Sensitivity Analyses 


C.2 Fourth Screener Package UAA Addresses 


About 30 of the sampled addresses ended up having their fourth NHES screener package 
returned as undeliverable even though none of their prior screener packages had been 
returned as such. We conducted an exploratory analysis of the characteristics of these 
addresses, with the purpose of determining whether fourth-screener-package-UAA-only 
addresses were more similar to addresses that did not have any UAA outcomes (and thus 
should be included in the nonrespondent results presented in this report) or to those that 
had inconsistent UAA outcomes for the first three screener packages (and thus should be 
included in the UAA address results presented in this report). Due to the small number of 
addresses fourth-screener-package-UAA-only cases, these results are not shown in tables. 


Key takeaway: Fourth-screener-package-UAA-only cases were not consistently more 
similar to either the inconsistent UAA or no UAA groups. As a result, we decided to include 
fourth-screener-package-UAA-only cases in the UAA analyses (with the inconsistent UAA 
addresses) because we felt this was most consistent with their final NHES:2019 status. 


Methods: We conducted the same comparisons as discussed for the first sensitivity analysis, 
except that we did not conduct statistical significance testing due to the small number of 
cases that only had the fourth screener mailing returned as undeliverable. We also removed 
the late respondents before doing this analysis because our first analysis showed that late 
respondents were systematically different from nonrespondent addresses. These tables are 
not included in the appendix because of small sample sizes. 


Results: Overall, we found that these cases were not consistently more similar to either the 
inconsistent UAA or no UAA groups across the full set of compared characteristics. For 
example, they were more similar to inconsistent-UAA addresses in terms of observed 
residential occupancy status but more similar to non-UAA addresses in terms of the 
education of the head of household variable on the NHES sampling frame. For other 
variables, like the route type variable on the NHES sampling frame, they were not very 
similar to either of the other groups. 
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This appendix provides additional details about the results of subgroup analyses. The goal 
of these analyses was to assess whether key themes in the interview and observation 
findings discussed in earlier chapters varied by interview participant or address 
characteristics. Section D.1 presents interview subgroup analyses, while section D.2 presents 
observation subgroup analyses. 


D.1 Interview Subgroup Analyses 


In this section, we discuss the findings of subgroup analyses related to interview 
participants’ reported engagement with the NHES:2019 screener-phase mailings (see 
section 5.2.2 for the overall findings). A series of bivariate analyses were conducted using 16 
auxiliary variables, including (1) interview participant self-reported characteristics (e.g., 
age, gender, race/ethnicity), (2) observation findings (e.g., structure type, mail access type), 
and (3) NHES:2019 and qualitative nonresponse study paradata (e.g., final screener 
response status, interview language). Because of the relatively small number of interview 
participants, the focus was on general patterns and statistical testing was not conducted. 


D.1.1 Recalling Mailings 


Seventy-seven percent of interview participants recalled at least one mailing (see table 5.1 
in chapter 5). Overall, there did not tend to be large differences in the recall rate by 
participant characteristics. However, some subgroups stood out as being more or less likely 
than others to remember at least one mailing. For example, the recall rate decreased as the 
number of adults in the household increased. Eighty-eight percent of participants who were 
the only adult in their household remembered at least one mailing, compared to 63 percent 
of those who lived in households with three or more adults. In addition, the recall rate was 
lower for participants for whom observers had not been able to locate a mailbox; only 60 
percent of them recalled at least one mailing, compared to 74 to 86 percent of participants 
for whom various mailbox types had been observed. Additional subgroup differences of note 
include: 


e Age: Only 33 percent of participants who were ages 18-24 remembered at least one 
mailing, compared to 71 to 89 percent of participants in other age groups.8 


e Educational attainment: Participants with graduate degrees were particularly likely 
to remember at least one mailing; all of them did so, compared to 73 to 76 percent of 
participants in the other educational attainment subgroups. 


e Employment status: Eighty-one percent of employed participants remembered at 
least one mailing, compared to 67 percent of participants who were not employed. 


8 We considered the possibility that was driven by participants in the 25-54 age group feeling that the mailings were more relevant to 
them (since participants in this age range would be most likely to have children who are eligible for the topical surveys). However, 
participants who had children living with them were no more likely to remember at least one mailing than those who did not have 
children living with them. 
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D.1.2 Opening Mailings 


Eighty percent of interview participants who remembered at least one mailing reported that 
they also had opened one or more mailings (see table 5.2 in chapter 5). There was some 
subgroup variation in the rate at which participants reported doing so. For example, in line 
with expectations, participants from households that ended up responding to the NHES were 
more likely to report opening at least one mailing (100 percent did so) than were 
participants from households who did not end up responding (75 percent did so). In 
addition, 94 percent of participants with children living in the home reported opening at 
least one mailing, compared to only 68 percent of participants who did not have children 
living in the home. Additional subgroup differences of note include: 


e Race/ethnicity and language spoken at home: Eighty-nine percent of Hispanic 
participants reported opening at least one mailing, compared to 72 percent and 73 
percent of White and Black participants. respectively. A similar pattern was observed 
for participants who lived in households where Spanish was the language most often 
spoken by adults (90 percent), compared to participants who lived in households 
where English was the language most often spoken by adults (76 percent). 


e Household income: As household income increased, so did the rate at which 
participants reported having opened at least one mailing (91 percent of participants 
with household incomes over $100,000 versus 75 percent of those with household 
incomes less than $30,000). In addition, only 67 percent of participants who declined 
to share their household income opened at least one mailing. 


e Observed structure type: Participants who lived in attached units were more likely 
to report opening at least one mailing (100 percent) than were participants who lived 
in single-unit homes or apartments (80 percent and 74 percent, respectively). 


e Observed mail access type: Only 67 percent of participants who received their mail 
through a mailbox, slot or room in a multi-unit building reported opening at least one 
mailing, compared to 83 to 88 percent of participants with other mail access types. 


e Household size: Participants living in households with two adults were more likely 
to report opening at least one mailing (97 percent). In comparison, only 58 percent of 
participants who lived in households with 3 or more adults reported opening at least 
one mailing. 


e Study site: Ninety-one percent of participants from the California site reported 
opening at least one mailing, compared to 71 percent of participants from the Texas 
site. 


D.1.3 Choosing Not to Respond 


Among interview participants who opened at least one mailing, 56 percent chose not to 
complete the survey—that is, they rejected or discarded the mailings (see table 5.3 in 
chapter 5). There was again some variation by subgroup in the likelihood of this outcome. 
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For example, 80 percent of participants who were age 65 and older chose not to complete 
the survey, compared to less than 60 percent of participants who were ages of 35 to 64. In 
addition, participants who only had home internet access through a phone or tablet were 
more likely to choose not to complete the survey (73 percent) than were participants who 
had home internet access through a computer (49 percent). Additional subgroup differences 
of note include: 


e Race/ethnicity: Sixty-nine percent of Black participants chose not to complete the 
survey, compared to 46 percent and 50 percent of White and Hispanic participants, 
respectively. 


e Educational attainment: The less education a participant had, the more likely he or 
she was to choose not to complete the survey; 67 percent of participants who had a 
high school education or less chose not to complete the survey versus 50 percent of 
participants with some college or a bachelor’s degree and 33 percent of participants 
with a graduate degree. 


e Structure type: Forty-six percent of participants who lived in single-unit structures 
chose not to complete the survey, compared to 63 to 64 percent of participants who 
lived in other structure types. 


e Household income: Seventy-five percent of participants whose household an 
income was between $30,001 and $60,000 chose not to complete the survey, 
compared to 50 percent or less of participants in each of the other income brackets. 
Sixty-seven percent of participants who refused to provide their household income 
did not complete the survey. 


D.1.4 Saving Mailings for Later 


Among interview participants who opened at least one mailing, 19 percent saved the 
materials for later or to give to someone else (see table 5.3 in chapter 5). There were again 
some subgroup differences in the likelihood of doing so. For example, about 43 percent of 
participants whose interview was conducted in Spanish saved the mailings, compared to 17 
percent of participants whose interview was conducted in English. A similar pattern was 
observed based on the language spoken most often by adults in the household. Additional 
subgroup differences of note include: 


e Age: Fifty percent of participants who were ages 55-64 saved the mailings, compared 
to 20 percent of participants who were ages 25-34. 


e Employment status: Thirty-one percent of participants who were not employed for 
pay saved the mailings, compared to 14 percent of participants who were employed 
for pay. 


e Household size: Forty percent of participants in single-adult households saved the 
mailings, compared to just 10 percent of participants in households with two adults. 
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e Structure type: Thirty-six percent of participants who lived in apartments saved the 
mailings, compared to 14 percent of participants who lived in single-unit structures. 


D.1.5 Choosing to Respond 


Finally, among interview participants who opened at least one mailing, 25 percent of them 
responded to the survey (see table 5.3 in chapter 5). There were again some subgroup 
differences in the likelihood of doing so. For example, participants with higher educational 
attainment responded at a higher rate than those with lower educational attainment; 50 
percent of participants with a graduate degree responded, compared to 13 percent of 
participants with a high school degree or less. In addition, participants with higher 
household incomes were more likely to respond than were those with lower household 
incomes; 50 percent of the participants with household incomes over $100,000 responded, 
compared to 25 percent of participants with household incomes between $30,001 and 
$60,000. Additional subgroup differences of note include: 


e Race/ethnicity: Thirty-nine percent of White participants responded to the survey, 
compared to 19 percent of Black participants. 


e Home internet access: Participants who only had home internet access via a phone 
or tablet responded at a lower rate (none of them did so) than did those who had 
home internet access via a computer (33 percent of them did so). 


e Household size: Thirty-three percent of participants who lived in two-adult 
households responded to the survey, compared to 13 percent of participants in 
single-adult households. 


e Study site: Thirty-one percent of participants from the Ohio site responded to the 
survey, compared to 17 percent of participants from the Texas site. 


D.2 Observation Subgroup Analyses 


In this section, we discuss the subgroup analyses that were conducted for the following 
observation outcomes: address characteristics, household characteristics, and agreement 
with frame variables. Subgroup analyses were conducted using more than 20 auxiliary 
variables, including (1) variables available on the NHES:2019 sampling frame (e.g., age, 
gender, or race of the head of household), (2) variables appended to the frame file from other 
publicly available federal government sources (e.g., tract poverty rate, percentage of 
households in Census block that include a child), and (3) NHES:2019 paradata (e.g., bilingual 
mailing status). For each observation outcome, we began by conducting a series of bivariate 
relationships; we used chi-square tests to identify statistically significant relationships. In 
most cases, we identified a considerable number of statistically significant bivariate 
relationships. Therefore, for each of the observation outcomes, we next conducted a 
multivariate logistic regression where the dependent variable was the observation outcome, 
and the independent variables were the same variables that had been included in the 
bivariate analyses. We identified statistically significant predictor variables using Wald joint 
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significance tests. All analyses in this section are limited to nonrespondent addresses that 
did not have UAA outcomes. 


D.2.1 Address Characteristics 


As part of the observation component of the study, observers collected information about 
the addresses at which nonrespondents lived: structure type and mail access type. For the 
overall results for these outcomes, see section 3.3.1 (structure type) and section 4.1.2 (mail 
access type). 


Structure type 


For the subgroup analyses, we used a collapsed version of structure type (single-unit, 
attached, apartment).? As shown in table D.1, almost all of the bivariate relationships were 
statistically significant. The multivariate analysis showed that 9 of the 21 independent 
variables included in the model were significant predictors of structure type (see table D.2). 
The most notable subgroup differences were for frame variables that also measured 
structure type. For example, addresses classified on the frame as multi-unit were particularly 
likely to be observed to be apartments (87 percent of addresses noted as multi-unit versus 
3 percent of cases noted as single-unit), while cases classified on the frame as single-unit 
were particularly likely to be observed to be single-unit structures (83 percent of addresses 
noted as single-unit versus 3 percent of addresses noted as multi-unit). The same patterns 
were seen for the route type variable on the frame. Additional significant predictors in the 
multivariate analysis include: 


e Gender of head of household: Addresses where the head of household was male 
were more likely to be single-unit structures (64 percent of those with a male head of 
household versus 51 percent of those with a female head of household), while those 
where the head of household was female were more likely to be attached structures 
(20 percent of those with a female head of household versus 7 percent of those with 
a male head of household). 


e Education of head of household: Addresses where the head of household had a 
graduate degree were particularly likely to be single-unit structures (81 percent 
versus 61 percent or less of other education subgroups) and particularly unlikely to 
be observed to be apartments (11 percent versus 27 percent or more of other 
education subgroups). 


e Household income: As household income increased, addresses became more likely 
to be single-unit structures (for example, 63 percent of addresses with household 
incomes of $100,000 or higher versus 48 percent of addresses with household 
incomes less than $50,000). The opposite pattern was seen for apartments (for 
example, 23 percent of addresses with household incomes of $100,000 or higher as 
opposed to 40 percent of addresses with household incomes less than $50,000). 


9 Structure type could not be determined for a very small number of addresses. They were excluded from this analysis. 
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e Number of adults in household: Addresses with only one adult living in them were 
particularly likely to be apartments (45 percent of addresses with one adult versus 9 
percent of addresses with two adults) and were particularly unlikely to be single-unit 
structures (40 percent of addresses with one adult versus 81 percent of addresses 
with 2 adults and 90 percent of addresses with three or more adults). 


e Home tenure: Addresses that were inhabited by their owner were particularly likely 
to be single-unit structures (79 percent of addresses inhabited by their owner versus 
26 percent of rented addresses), while addresses that were rented were particularly 
likely to be apartments (57 percent of rented addresses versus 11 percent of owned 
addresses). 


e Region: There were mild differences in the distribution of structure type by region. 
Addresses in the Midwest were somewhat more likely than those in other regions to 
be single-unit structures (for example, 59 percent of Midwestern addresses versus 42 
percent of Northeastern addresses) and somewhat less likely than those in other 
regions to be apartments (for example, 29 percent of Midwestern addresses versus 
44 percent of Southern addresses). In addition, addresses in the Northeast were 
slightly more likely than those in other regions to be attached structures (22 percent 
versus 13 percent or less of addresses in other regions). 


e LowResponse Score: Addresses in block groups with lower scores (i.e., block groups 
with higher expected mail return rates to the Decennial Census) were more likely to 
be single-unit structures than were addresses in block groups with higher scores (i.e., 
block groups with lower expected mail return rates to the Decennial Census) (85 
percent of first quartile addresses versus 27 percent of fourth quartile addresses). 
The opposite pattern was seen for apartments (6 percent of first quartile addresses 
were apartments, versus 59 percent of fourth quartile addresses). 


In addition, addresses that were missing information (for gender of head of household, 
education of head of household, household income, number of adults in the household, home 
tenure) were particularly likely to be apartments. For example, about 70 percent of 
addresses that were missing information about the education of the head of household were 
observed to be apartments, as opposed to 32 percent or less of addresses in the other 
education subgroups. We also observed this pattern for several of the bivariate analyses (see 
chapter 7 for additional analysis of the observed and self-reported characteristics of 
addresses that were missing information for frame variables). 
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Table D.1. Percentage distribution of observed structure type for observed, 
occupied, residential, nonrespondent addresses, by selected 
characteristics: 2019 


Observed structure type! 


Number of Chi-square 
Selected characteristics addresses Single-unit | Attached Apartment statistic 
Total 530 49.9 12.9 37.2 t 
NHES:2019 screener mailings language 17.41 * 
All bilingual mailings (English and Spanish) 200 49.5 15.8 34.7 
Mix of bilingual mailings and English- only 
mailings 270 45.3 11.3 43.4 
All English-only mailings 60 71.0 9.7 19.4 
Age of head of household2 88.94 * 
18-34 50 50.0 12.0 38.0 
35-44 80 60.5 9.2 30.3 
45-54 90 73.9 13.6 12.5 
55-64 70 67.1 9.6 23.3 
65 and older 60 56.7 15.0 28.3 
Missing 180 24.7 14.8 60.4 
Gender of head of household2 39.91 * 
Male 160 64.4 6.8 28.8 
Female 160 51.2 19.8 29.0 
Missing 200 37.3 12.3 50.5 
Education of head of household2 83.35 * 
Less than high school credential 120 51.7 16.1 32.2 
High school credential 80 60.0 10.0 30.0 
Some college 110 56.3 17.0 26.8 
Bachelor’s degree 70 60.9 7.3 31.9 
Graduate degree 40 81.1 8.1 10.8 
Missing 110 17.7 12.4 69.9 
Race of head of household? 73.75 * 
White, non-Hispanic 170 63.2 11.1 25.7 
Black, non-Hispanic 90 52.3 18.6 29.1 
Hispanic 120 59.2 11.7 29.2 
Other race, non-Hispanic 40 47.2 16.7 36.1 
Missing 120 19.8 11.2 69.0 
Household income2 72.53 * 
Less than $50,000 220 48.4 12.0 39.6 
$50,000-$74,999 70 63.1 13.9 23.1 
$75,000-$99,999 60 57.9 12.3 29.8 
$100,000 or higher 130 62.8 14.0 23.3 
Missing 60 6.6 13.1 80.3 
Household flagged as having children2 25.60 * 
Yes 140 65.9 14.8 19.3 
No 390 44.4 12.2 43.4 


See notes at end of table. 
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Table D.1. Percentage distribution of observed structure type for observed, 
occupied, residential, nonrespondent addresses, by selected 
characteristics: 2019—Continued 


Observed structure type! 


Number of Chi-square 
Selected characteristics addresses Single-unit | Attached Apartment statistic 
Number of adults in household2 147.91 * 
1 adult 300 39.9 15.0 45.2 
2 adults 120 81.0 10.3 8.6 
3 adults or more 50 90.2 5.9 $ 
Missing 60 6.6 13.1 80.3 
Phone number available2 48.01 * 
Yes 330 61.2 12.2 26.6 
No 200 31.7 13.9 54.5 
Route type2 385.54 * 
Street 330 77.5 16.5 6.0 
High rise 200 2.6 6.7 90.8 
Dwelling type2 406.35 * 
Single-unit 310 82.7 14.4 2.9 
Multi-unit 220 2.8 10.6 86.6 
Home tenure2 208.68 * 
Own 270 78.6 10.0 11.4 
Rent 180 25.7 17.7 56.6 
Missing 80 Tit 12.1 80.7 
Urbanicity2 53.55 * 
Urban 240 34.0 13.6 52.3 
Suburban 270 60.6 12.6 26.8 
Town $ $ $ $ 
Rural $ $ $ $ 
Region2 16.04 * 
Northeast 80 41.6 22.1 36.4 
South 220 45.7 10.5 43.8 
Midwest 150 59.1 11.7 29.2 
West 80 51.9 12.7 35.4 
Race/ethnicity stratum2 5.66 
25% or more Black 140 45.3 14.6 40.2 
40% or more Hispanic 160 49,7 9.0 41.3 
Other 240 52.7 14.4 32.9 
Tract poverty rate? 16.15 * 
Less than 20% 360 55.5 13.0 31.6 
20% or more 170 38.5 12.6 48.9 
Percent of households in Census block that 
include a child? 24.93 * 
First quartile 140 36.4 10.0 53.6 
Second quartile 110 51.3 15.0 33.6 
Third quartile 120 55.1 17.0 28.0 
Fourth quartile 160 57.0 10.8 32.3 


See notes at end of table. 
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Table D.1. Percentage distribution of observed structure type for observed, occupied, 
residential, nonrespondent addresses, by selected characteristics: 2019— 


Continued 
Observed structure type! 
Number of Chi-square 
Selected characteristics addresses Single-unit | Attached Apartment statistic 
Percent of persons in Census block that speak 
a language other than English? 21.82 * 
First quartile 40 68.2 15.9 15.9 
Second quartile 90 60.0 15.3 24.7 
Third quartile 160 50.9 9.8 39.3 
Fourth quartile 240 42.2 13.5 44.3 
Percent of persons in Census block without 
a high school diploma or the equivalent? 7.82 
First quartile 120 47.1 12.6 40.3 
Second quartile 100 59.4 14.9 25.7 
Third quartile 120 50.9 11.0 38.1 
Fourth quartile 190 46.1 13.1 40.8 
Low Response Score* 117.36* 
First quartile 100 85.4 8.7 5.8 
Second quartile 90 63.6 15.9 20.5 
Third quartile 120 51.7 11.9 36.4 
Fourth quartile 220 26.9 13.7 59.4 
Residential high-speed internet per 1000 
households® 6.97 
600 or less 120 42.2 14.1 43.8 
601-800 200 49.7 10.8 39.5 
801 or more 210 54.5 14.1 31.5 
tNot applicable. 
+Reporting standards not met. There are too few cases for a reliable estimate. 
*p < 0.05. 


1Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise apartments. Structure type was not 
able to be determined for a very small number of cases, and these cases have been excluded from this analysis. 

2These characteristics are based on variables available on the NHES:2019 sampling frame. 

3These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest 
prevalence of the characteristic in question and those that are in the fourth quartile are those with the highest prevalence. 

4The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. A higher score corresponds to a lower expected mail return rate. This variable was not available for a very small number of cases, and these cases 
have been excluded from this analysis. 

5This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The structure type observation was not collected for addresses that could not be observed (e.g., cannot locate the address). Addresses that ended up 
responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from this analysis. 
Addresses that were observed to be nonresidential were excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded 
to one decimal place but have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 
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Table D.2. Wald joint significance tests from multinomial logistic regression 
predicting structure type, by independent variables: 2019 


Independent variables Wald test statistic 
NHES:2019 bilingual screener mailings status 3.83 
Age of head of household! 10.21 
Gender of head of household! 10.48 * 
Education of head of household! 21.62 * 
Race of head of household! 9.42 
Household income! 17.84 * 
Whether household flagged as having children! 0.60 
Number of adults in household? 22.49 * 
Whether phone number available! 1.42 
Route type! 12.57 * 
Dwelling type! 18.54* 
Home tenure! 26.89 * 
Urbanicity12 7.69 
Region! 22.33 * 
Race/ethnicity stratum3 3.44 
Tract poverty rate? 1.07 
Percent of households in Census block that include a child3 5.63 
Percent of persons in Census block that speak a language other than English3 9.30 
Percent of persons in Census block without a high school diploma or the equivalent? 10.00 
Low Response Score* 15.39 * 
Residential high-speed internet per 1000 households® 2.56 
*p <.05. 


1These variables are available on the NHES:2019 sampling frame. 

2This variable originally had four categories (urban, suburban, town, rural), but the town and rural categories had to be collapsed for this analysis due to 
sample size constraints. 

3These characteristics are based on American Community Survey (2013-2017) five-year estimates. 

4The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

5This variable is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The dependent variable is a collapsed version of the observed structure type with three categories—single-unit, attached, and apartment. "Single- 
unit" is the reference category. Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise 
apartments. Missing values on the independent variables are treated as a separate category. Structure type observations were not collected for addresses 
that could not be observed (e.g., cannot locate the address). Structure type was not able to be determined for a very small number of cases, and these cases 
have been excluded from this analysis. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed 
(UAA) NHES:2019 mailing were excluded from this analysis. Addresses that were observed to be nonresidential were excluded from this analysis. The 
pseudo R? value for the model is 0.70. The rounded sample size is 530. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 


Mail access type 


For the subgroup analyses, we used a slightly collapsed version of mail access type (mail slot 
or mailbox attached to the home; mailbox at the end of the driveway, across the street, or at 
the end of the road; mailbox, slot, or room in multi-unit building; no mailbox or slot in view; 
could not determine). Like the structure type analyses, nearly all of the bivariate 
relationships were statistically significant (see table D.3). The multivariate analysis showed 
that 5 of the 22 independent variables included in the model were significant predictors of 
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mail access type (see table D.4).1° The most notable differences were for variables that 
measured structure type. For example, while two-fifths of addresses flagged on the frame as 
being high-rise route types had mailboxes, slots, or rooms in multi-unit buildings (40 
percent), this mail access type was very rare for addresses with street route types. While 
observers were not able to determine the mail access type for about a quarter of addresses 
with high-rise route types (22 percent), this outcome was very rare for addresses with a 
street route type (2 percent). The same patterns were seen for observed structure type. 
Additional significant predictors in the multivariate analysis include: 


e Race/ethnicity stratum:!! Addresses in the Black oversample stratum were more 
likely to have, mail slots or mailboxes attached to the home than they were to have 
other mail access types (48 percent versus 20 percent or less for other types). In 
contrast, addresses in the other strata (Hispanic oversample and Other), were more 
evenly split between two mail access types: (1) mail slot or mailbox attached to the 
home and (2) mailbox at the end of the driveway, across the street, or at the end of 
the road (39 percent and 31 percent, respectively for the Hispanic oversample 
stratum, and 30 and 37 percent, respectively for the Other stratum). 


e Percentage of persons in Census block without a high school diploma or the 
equivalent: Addresses in Census blocks with lower percentages of persons without 
a high school diploma or the equivalent (first and second quartiles) were more likely 
than addresses with higher percentages of such persons (third and fourth quartiles) 
to have a mailbox at the end of the driveway, across the street, or at the end of the 
road (about two-fifths versus about a quarter). Addresses in Census blocks with the 
lowest percentages of persons without a high school diploma or the equivalent (first 
quartile) also were less likely than other addresses to have a mail slot or mailbox 
attached to their home (24 percent versus 38 percent or more of addresses in other 
quartiles). 


e Low Response Score: Addresses in block groups with the lowest scores were more 
likely than other addresses to have a mailbox at the end of the driveway, across the 
street, or at the end of the road (57 percent in the first quartile versus less than a third 
of addresses in other quartiles). In addition, observers were more likely to be unable 
to determine mail access type for addresses in block groups with a higher score (first 
and second quartiles) than for those in block groups with a lower score (third and 
fourth quartile) (about one-seventh versus less than 1 in 20). 


In addition, observers also tended to be more likely to be unable to determine the mail access 
type for addresses that were missing information on the frame (e.g., education and race of 
head of household, household income, number of adults in the household, home tenure). For 
example, observers could not determine the mail access type for a quarter of the addresses 
that were missing household income information on the frame (26 percent), as opposed to 


10 In addition to the 21 variables included in the structure type analysis, we also included the mail access type observation here. 

11 Addresses in the Black oversample stratum for the NHES sample design are those in Census tracts with 25 percent or more Black 
persons. Addresses in the Hispanic oversample stratum are those with 40 percent or more Hispanic persons and not 25 percent or more 
Black persons. All other addresses are in the “other” stratum. 
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less than 10 percent of addresses for which household income information was available on 
the frame. 
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Table D.3. Percentage distribution of observed mail access type for observed, occupied, residential, nonrespondent addresses, 
by selected characteristics: 2019 


Selected characteristics 
Total 

Observed structure type2 
Single-unit 
Attached 
Apartment 


NHES:2019 screener mailings language 
All bilingual mailings (English and Spanish) 
Mix of bilingual mailings and English-only mailings 


All English-only mailings 
Age of head of household? 
18-34 
35-44 
45-54 
55-64 
65 and older 
Missing 
Gender of head of household? 
Male 
Female 
Missing 
Education of head of household? 
Less than high school credential 
High school credential 
Some college 
Bachelor's degree 
Graduate degree 
Missing 


See notes at end of table. 


Number of 
addresses 


530 


260 
70 
200 


200 
270 
60 


50 
80 
90 
70 
60 
180 


160 
160 
210 


120 
80 
110 
70 
40 
110 


Mail slot or 


mailbox 


attached to 


the home 
37.3 


47.0 
54.4 
18.3 


36.6 
35.3 
47.6 


39.2 
43.4 
39.8 
42.5 
38.3 
30.6 


39.6 
44.4 
29.8 


45.4 
33.8 
47.3 
31.9 
35.1 
25.4 


Observed mail access type 


Mailbox at the end 
of the driveway, 
across the street, 
or at the end 

of the road 


30.7 


45.5 
33.8 
9.6 


31.2 
30.1 
31.8 


29.4 
36.8 
46.6 
35.6 
35.0 
17.5 


36.0 
29.0 
27.8 


30.3 
37.5 
28.6 
39.1 
54.1 
15.8 
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Mailbox, slot 
or room in 
multi-unit 
building 


14.9 


No mailbox or 
slot in view 


7.5 


6.1 
8.8 
9.1 


5.9 
9.4 
4.8 


Could not 
determine 


9.6 


1.5 


Chi-square 
statistic 


Tt 
269.29 * 


8.36 


65.85 * 


26.23 * 


84.12 * 
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Table D.3. Percentage distribution of observed mail access type for observed, occupied, residential, nonrespondent 
addresses, by selected characteristics: 2019—Continued 


Selected characteristics 


Race of head of household? 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other race, non-Hispanic 
Missing 

Household income? 
Less than $50,000 
$50,000-$74,999 
$75,000-$99,999 
$100,000 or higher 
Missing 

Household flagged as having children? 
Yes 
No 

Number of adults in household? 
1 adult 
2 adults 
3 adults or more 
Missing 

Phone number available? 
Yes 
No 


Number of 


addresses 


170 
90 
120 
40 
120 


220 
70 
60 

130 
60 


140 
400 


300 
120 
50 
60 


330 
200 


Mail slot or 


mailbox 


attached to 


the home 


35.1 
55.8 
38.8 
30.6 
27.4 


39.6 
40.9 
43.9 
36.4 
21.0 


46.7 
34.1 


37.9 
37.6 
52.9 
21.0 


39.8 
33.2 


Mailbox at the end 
of the driveway, 
across the street, 
or at the end 

of the road 


42.7 
17.4 
37.2 
36.1 
14.5 


25.4 
40.9 
31.6 
43.4 
11.3 


37.8 
28.3 


27.2 
46.2 
39.2 
11.3 


35.9 
22.3 


Mailbox, slot 
or room in 
multi-unit 
building 


11.7 
9.3 
9.9 

19.4 

27.4 


16.6 
6.1 
14.0 
7.8 
33.9 


No mailbox or 
slot in view 


4.7 
11.6 
7.4 
8.3 
8.6 


10.1 


6.4 
9.4 


Could not 
determine 


5.9 
5.8 
6.6 


+ 
22.2 


6.7 
14.4 


Observed mail access type! 


Chi-square 
statistic 


83.25 * 


71.44 * 


23.34 * 


84.66 * 


25.86 * 


See notes at end of table. 
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Table D.3. Percentage distribution of observed mail access type for observed, occupied, residential, nonrespondent 
addresses, by selected characteristics: 2019—Continued 


Selected characteristics 
Route type? 
Street 
High rise 
Dwelling type? 
Single-unit 
Multi-unit 
Home tenure? 
Own 
Rent 
Missing 
Urbanicity3 
Urban 
Suburban 
Town 
Rural 
Region? 
Northeast 
South 
Midwest 
West 
Race/ethnicity stratum* 
25% or more Black 
40% or more Hispanic 
Other 
Tract poverty rate*+ 
Less than 20% 


20% or more 
See notes at end of table. 


Number of 


addresses 


340 
200 


310 
220 


270 
180 
80 


Mail slot or 


mailbox 


attached to 


the home 


49.7 
15.9 


47.9 
22.0 


42.7 
36.6 
21.4 


34.6 
43.1 


Mailbox at the end 
of the driveway, 
across the street, 
or at the end 

of the road 


42.0 
11.3 


44,7 
10.6 


42.7 
21.1 
11.9 


23.6 
32.7 


Mailbox, slot 
or room in 
multi-unit 
building 


No mailbox or 
slot in view 


apyA 
10.8 


5.4 
10.6 


6.6 
8.6 
8.3 


Observed mail access type! 


Could not 
determine 


2.4 
22.1 


12.5 


Chi-square 
statistic 


260.38 * 


236.84 * 


112.93 * 


52.49 * 


45.58 * 


19.52 * 


10.24 * 
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Table D.3. Percentage distribution of observed mail access type for observed, occupied, residential, nonrespondent 
addresses, by selected characteristics: 2019—Continued 


Observed mail access type! 
Mailbox at the end 


Mail slot or of the driveway, Mailbox, slot 
mailbox across the street, or room in 
Number of attached to or at the end multi-unit Nomailboxor Couldnot Chi-square 
Selected characteristics addresses the home of the road building slotinview determine statistic 
Percent of households in Census block that 35.12 * 
include a child* 
First quartile 140 36.2 17.7 24.8 6.4 14.9 
Second quartile 110 41.6 30.1 14.2 6.2 8.0 
Third quartile 120 39.8 37.3 9.3 8.5 5.1 
Fourth quartile 160 33.3 37.7 10.7 8.8 9.4 
Percent of persons in Census block that speak a 
language other than English* 20.34 
First quartile 50 51.1 31.1 6.7 6.7 44 
Second quartile 90 48.2 29.4 8.2 $ 11.8 
Third quartile 160 30.1 34.4 16.6 8.6 10.4 
Fourth quartile 240 35.7 28.6 17.6 8.8 9.2 
Percent of persons in Census block without a high 
school diploma or the equivalent* 37.44 * 
First quartile 120 23.5 36.1 19.3 6.7 14.3 
Second quartile 100 38.6 40.6 6.9 5.0 8.9 
Third quartile 120 44.9 26.3 18.6 $ 7.6 
Fourth quartile 190 40.4 24.9 14.0 12.4 8.3 
Low Response Score®> 70.85 * 
First quartile 100 36.9 57.3 t t 1.9 
Second quartile 90 46.6 29.5 12.5 6.8 4.5 
Third quartile 120 36.4 24.6 17.8 7.6 13.6 
Fourth quartile 220 34.4 21.7 20.4 10.4 13.1 
Residential high-speed internet per 1000 households¢é 32.96 * 
600 or less 120 44.7 22.0 17.1 8.9 7.3 
601-800 200 46.2 24.1 13.9 5.1 10.8 
801 or more 210 24.9 41.8 14.6 8.9 9.9 


tNot applicable. 

#Rounds to zero. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

*p < 0.05. 

1A few cases where mail was received in more than one way were categorized under "mail slot or mailbox attached to the home" since all the ways mail was received fit that category. 

2Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise apartments. Structure type was not able to be determined for a very small number of cases, and 
these cases have been excluded from this analysis. 

3These characteristics are based on variables available on the NHES:2019 sampling frame. 
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4These characteristics are based on American Community Survey (2013-2017) five year estimates. Cases in the first quartile were those with the lowest prevalence of the characteristic in question and those that are in 
the fourth quartile are those with the highest prevalence. 


5The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial Census. A higher score corresponds to a lower expected mail 
return rate. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

6This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The mail access type observation was not collected for addresses that could not be observed (e.g., cannot locate the address) or were observed to be nonresidential or permanently vacant. Addresses that ended 
up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are 
rounded to one decimal place but have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; and U.S. Department of Commerce, American Community Survey (ACS), 
2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 2017. 
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Table D.4. Wald joint significance tests from multinomial logistic regression 
predicting mail access type, by independent variables: 2019 


Independent variables Wald test statistic 
Observed structure type 21.48 * 
NHES:2019 bilingual screener mailings status 4.94 
Age of head of household! 17.35 
Gender of head of household! 14.59 
Education of head of household! 20.33 
Race of head of household! 22.35 
Household income! 21.40 
Whether household flagged as having children 4.92 
Number of adults in household 13.12 
Whether phone number available 0.61 
Route type! 23.02 * 
Dwelling type! tT 
Home tenure! 1.36 
Urbanicity13 6.81 
Region! 18.39 
Race/ethnicity stratum* 17.48 * 
Tract poverty rate* 4.32 
Percent of households in Census block that include a child*+ 17.46 
Percent of persons in Census block that speak a language other than English* 7.26 
Percent of persons in Census block without a high school diploma or the equivalent* 23.00 * 
Low Response Score® 27.95* 
Residential high-speed internet per 1000 households® 12.45 
tNot applicable. 

*p<.05. 


1These variables are available on the NHES:2019 sampling frame. 

2Dwelling type was dropped from the model because all multi-unit dwellings had the same value for the dependent variable (mailbox, slot, or room in multi- 
unit building). 
3This variable originally had four categories (urban, suburban, town, rural), but the town and rural categories had to be collapsed for this analysis due to 
sample size constraints. 
4 These characteristics are based on American Community Survey (2013-2017) five-year estimates. 

5The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

6This variable is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The dependent variable is a collapsed version of the observed mail access type with five categories—mail slot or mailbox attached to the home; mailbox 
at the end of the driveway, across the street or at the end of the road; mailbox, slot, or room in multi-unit building; no mailbox or slot in view; and could not 
determine. "Could not determine" is the reference category. Missing values on the independent variables are treated as a separate category. The mail access 
type observation was not collected for addresses that could not be observed (e.g., cannot locate the address) or were observed to be nonresidential or 
permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 
mailing were excluded from this analysis. The pseudo R? value for the model is 0.43. The rounded sample size is 530. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 
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D.2.2 Household Characteristics 


Many of the household characteristic observations involved noting the presence of an 
indicator (e.g., the presence of children, patriotism) (see section 3.3 for the overall results). 
For these outcomes, we only conducted subgroup analysis when the characteristic was 
observed in at least 20 percent of addresses. For the household characteristic observations 
discussed in this section, in addition to the analyses described in earlier sections of this 
appendix, we also conducted a second set of subgroup analyses to determine if there were 
statistically significant relationships between the various observation variables. For each of 
the characteristics discussed below, we conducted (1) 10 additional observation subgroup 
bivariate analyses and (2) a second multinomial regression for each characteristic that 
included the observation variables as independent variables. For several of these 
observation variables, structure type was a significant predictor of the presence of 
household characteristics, with apartments being less likely than single-unit homes to have 
observable evidence of the household characteristics. This is likely due to the more limited 
outdoor space that can be directly attributed to individual apartments (e.g., limited to 
balcony, unit door) compared to single-unit homes (e.g., front/back/side yard, driveway 
(including cars)). 


Presence of children 


As compared to the address characteristic observations discussed in section D.2.1, fewer of 
the bivariate relationships were statistically significant: 7 of the 21 auxiliary variables and 5 
of the 10 observation variables (see tables D.5 and D.6 at the end of this section). However, 
as shown in tables D.7 and D.8 the end of this section, only two variables were statistically 
significant predictors of observing the presence of children in the multivariate models. 


e Race of head of household: The presence of children was observed less often for 
households where the frame indicated the head of household was Black (10 percent) 
than it was for households where the head of household was White or Hispanic (26 
percent each). 


e Outdoor living observation: Addresses that were observed to value outdoor living 
were more likely to be observed to have children living in them than addresses that 
were not observed to value outdoor living (50 percent versus 12). 


In addition, there were several indications that it was more challenging to observe household 
attributes in multi-unit addresses; this finding is repeated for several of the observation 
analyses discussed in this appendix. For example, addresses flagged on the frame as being 
high-rise addresses were observed to have children living in them less often than street 
addresses (10 percent versus 25 percent), and addresses flagged on the frame as being 
multi-unit addresses were observed to have children living in them less often than single- 
unit addresses (8 versus 27 percent). Similarly, addresses observed to be attached units or 
apartment were less likely to be observed to have children living in them (14 percent and 12 
percent, respectively) than were single-unit structures (27 percent). Finally, addresses 
observed to have a mailbox, slot, or room in a multi-unit building were observed to have 
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children living in them less often (7 percent) than were addresses that were observed to 
have a mail slot or mailbox attached to the home or to have a mailbox at the end of the 
driveway, across the street, or at the end of the road (22 percent and 29 percent, 
respectively). 


Privacy or security concerns 


There was a Statistically significant bivariate relationship between the privacy or security 
concerns observation and 11 of the 21 auxiliary variables and 4 of the 10 observation 
variables (see tables D.5 and D.6). In the multivariate models, three auxiliary variables and 
two observation variables were statistically significant predictors of observing privacy or 
security concerns (see tables D.7 and D.8). The most notable differences were for variables 
that measured structure type. For example, addresses that were observed to be single-unit 
structures were particularly likely to be observed to have privacy or security concerns, 
attached units were somewhat less likely to do so, and apartments were much less likely to 
do so (43 percent, 24 percent, and 6 percent, respectively). A similar pattern was observed 
for the route type variable on the sampling frame. Additional significant predictors in the 
multivariate analysis include: 


e Observed mail access type: Addresses whose mail access was attached to the home 
or was nearby were most likely to be observed to have privacy or security concerns 
(39 percent and 34 percent, respectively). 


e Region: Addresses in the Northeast were less likely than addresses in other regions 
to be observed to have privacy or security concerns (18 percent versus 30 percent or 
more of address in other regions). 


e Residential high-speed internet per 1000 households: Addresses in areas with 
the lowest rate of high-speed internet penetration were somewhat more likely to be 
observed to have privacy or security concerns (37 percent versus 31 percent or less 
of addresses in areas with higher rates of penetration). 


Outdoor living 


There was a Statistically significant bivariate relationship between the outdoor living 
observation and 7 of the 21 auxiliary variables and 5 of the 10 observation variables (see 
tables D.5 and D.6). In the multivariate models, only three variables (one auxiliary variable 
and two observation variables) were statistically significant predictors of the outdoor living 
observation (see tables D.7 and D.8). 


e Phone number available: Addresses that had a phone number available on the 
sampling frame were more likely than those that did not to be observed to value 
outdoor living (30 percent versus 13 percent). 


e Presence of children observation: Similar to the finding noted in the “presence of 


children” analysis, addresses where children were observed to live were also more 
likely to be observed to value outdoor living than were those addresses that were not 
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observed to have children living in them or where it was unclear whether or not 
children lived there (57 percent versus 16 percent). 


e Other outdoor decor observation: Addresses that were observed to have other 
outdoor decor were more likely to be observed to value outdoor living than were 
those addresses that were not observed to have other outdoor decor (41 percent 
versus 19 percent). 


Other outdoor decor 


There was a statistically significant bivariate relationship between the other outdoor decor 
observation and 10 of the 21 auxiliary variables and 7 of the 10 observation variables (see 
tables D.5 and D.6). In the multivariate models, 4 auxiliary variables and 4 observation 
variables were statistically significant predictors of the other outdoor decor (see tables D.7 
and D.8). Structure type was again a notable predictor of the presence of other outdoor 
decor; addresses that were observed to be single-unit structures were more likely to have 
outdoor decor (33 percent), while those observed to be apartments were less likely to do so 
(8 percent). It also appeared that some households were more likely overall to leave items 
of various types outside of their homes. For example, addresses that were observed to have 
indicators of several other household attributes also tended to be more likely to be observed 
to have other outdoor decor. For example, those addresses observed to value outdoor living 
were more likely to have other outdoor decor than those not observed to value outdoor 
living (41 percent versus 19 percent);those that had indicators of patriotism were more 
likely to have other outdoor decor than those without indicators of patriotism (82 percent 
versus 20 percent); and those that had welcoming decor were more likely to have other 
outdoor decor than those without welcoming decor (34 percent versus 24 percent). 
Additional significant predictors in the multivariate analysis include: 


e Education of head of household: Addresses where the frame indicated the head of 
the household had less than a high school credential or a bachelor’s degree were more 
likely to have other outdoor decor (33 percent and 36 percent, respectively) than 
other addresses (21 percent or less of addresses in other education subgroups). 


e Urbanicity: Addresses in suburban areas were somewhat more likely than those in 
urban areas to have outdoor decor (26 percent versus 20 percent). 


e Percentage of households in Census block that include a child: Addresses located 
in Census blocks in the second quartile in terms of the percentage of households with 
a child were more likely than other addresses to have outdoor decor (33 percent 
versus 24 percent or less of addresses in other quartiles). 


e Low Response Score: Addresses in Census blocks whose score was in the fourth 
quartile were the least likely to have other outdoor decor (15 percent). Addresses in 
Census blocks who score was in the second quartile were the most likely to have 
outdoor decor (42 percent). 
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Table D.5. Percentage of observed, occupied, residential, nonrespondent addresses with observed household attributes, 
by observed household attributes and selected characteristics: 2019 


Observed household attributes 
Privacy or security 


Presence of children concerns? Outdoor living? Other outdoor decor* 
Selected characteristics addresses Percentage _ statistic Percentage statistic Percentage statistic Percentage statistic 
Total 440 21.4 t 31.2 t 24.6 t 24.6 t 
NHES:2019 screener mailings language 3.36 0.47 4.18 1.10 
All bilingual mailings (English and Spanish) 170 17.3 30.4 21.4 22.6 
Mix of bilingual mailings and English-only 
mailings 220 23.0 30.9 24.4 24.9 
All English-only mailings 50 27.8 35.2 35.2 29.6 
Age of head of household5 12.48 * 10.02 3.14 5.74 
18-34 40 39.5 41.9 27.9 23.3 
35-44 70 26.9 32.8 23.9 17.9 
45-54 80 19.5 39.0 29.3 32.9 
55-64 60 19.0 30.2 27.0 23.8 
65 and older 50 19.2 32.7 25.0 28.9 
Missing 130 15.9 22.0 19.7 22.0 
Gender of head of household5 5.46 6.75 * 6.95 * 3.46 
Male 140 27.5 33.8 32.4 29.6 
Female 140 20.8 36.8 20.1 20.1 
Missing 150 16.3 23.5 21.6 24.2 
Education of head of household® 5.31 15.64 * 2.36 12.10 * 
Less than high school credential 110 22.4 33.6 25.2 32.7 
High school credential 70 23.2 27.5 21.7 20.3 
Some college 100 21.2 40.4 24.0 18.3 
Bachelor's degree 60 26.8 32.1 25.0 35.7 
Graduate degree 40 25.7 37.1 34.3 17.1 
Missing 70 11.8 13.2 22.1 20.6 
Race of head of household5 9.98 * 1121" 8.70 11.87 * 
White, non-Hispanic 150 26.0 33.3 32.0 28.7 
Black, non-Hispanic 80 10.3 39.7 15.4 10.3 
Hispanic 110 26.1 32.4 24.3 29.7 
Other race, non-Hispanic + + + + + 
Missing 70 16.9 15.5 21.1 22.5 


See notes at end of table. 
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Table D.5. Percentage of observed, occupied, residential, nonrespondent addresses with observed household attributes, 
by observed household attributes and selected characteristics: 2019—Continued 


Observed household attributes 


Privacy or security 


Presence of children concerns? Outdoor living? Other outdoor decor* 
Number of Chi-square Chi-square Chi-square Chi-square 
Selected characteristics addresses Percentage Statistic Percentage statistic Percentage statistic Percentage statistic 
Household income> 8.71 12.40 * 8.46 4.40 
Less than $50,000 180 21.2 30.7 21.2 25.7 
$50,000-$74,999 60 30.0 40.0 36.7 26.7 
$75,000-$99,999 50 18.0 28.0 24.0 18.0 
$100,000 or higher 110 23.7 36.0 27.2 28.1 
Missing 40 $ 8.3 13.9 13.9 
Household flagged as having children® 7.35 * 2.56 5.38 * 0.01 
Yes 130 29.7 36.7 32.0 24.2 
No 310 18.0 28.9 21.5 24.8 
Number of adults in household® 8.91 * 13.85 * 5.83 3.12 
1 adult 240 20.7 29.5 22.4 24.5 
2 adults 110 28.6 40.2 31.3 25.9 
3 adults or more 50 20.0 36.0 28.0 30.0 
Missing 40 $ 8.3 13.9 13.9 
Phone number available5 2.39 2.73 15.84 * 7.86 * 
Yes 290 23.5 33.8 30.4 28.7 
No 150 17.1 26.0 13.0 16.4 
Route type> 12.33 * 50.84 * 10.53 * 20.35 * 
Street 330 25.5 40.5 28.5 30.1 
High rise 110 9.7 44 13.3 8.9 
Dwelling type 18.80 * 48.33 * 10.26 * 26.43 * 
Single-unit 310 26.9 41.2 28.9 31.5 
Multi-unit 130 8.4 7.6 14.5 8.4 
Home tenure® 2.45 21.38 * 14.47 * 
Own 260 23.0 39.3 29.2 8.00 * 30.7 
Rent 140 21.5 23.0 20.0 18.5 
Missing 50 12.8 10.6 12.8 8.5 


See notes at end of table. 
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Table D.5. Percentage of observed, occupied, residential, nonrespondent addresses with observed household attributes, 
by observed household attributes and selected characteristics: 2019—Continued 
Observed household attributes 


Privacy or security 


Presence of children concerns? Outdoor living? Other outdoor decor* 
Number of Chi-square Chi-square Chi-square Chi-square 
Selected characteristics addresses Percentage statistic Percentage Statistic Percentage statistic Percentage statistic 
Urbanicity® 1.38 4.92 5.65 8.29 * 
Urban 180 18.8 27.3 20.5 19.9 
Suburban 240 23.0 34.7 26.8 25.9 
Town + + + + + 
Rural £ $ £ $ $ 
Region> 1.72 8.27 * 1.52 5.09 
Northeast 70 20.0 18.5 27.7 24.6 
South 190 21.3 30.3 22.3 21.3 
Midwest 120 19.4 38.7 24.2 24.2 
West 60 274 32.3 29.0 35.5 
Race/ethnicity stratum® 13.81 * 3.21 5.31 6.20 * 
25% or more Black 110 10.6 37.2 16.8 15.9 
40% or more Hispanic 130 30.2 31.8 25.6 27.1 
Other 200 218 274 28.4 27.9 
Tract poverty rate® 0.01 0.06 0.05 6.83 * 
Less than 20% 300 21.3 31.6 24.9 28.2 
20% or more 140 21.7 30.4 23.9 16.7 
Percent of households in Census block that 
include a child¢ 1.17 7.52 5.58 5.04 
First quartile 100 18.2 22.2 18.2 20.2 
Second quartile 100 20.4 27.6 31.6 32.7 
Third quartile 100 22.3 37.9 27.2 21.4 
Fourth quartile 140 23.7 35.3 22.3 24.5 
Percent of persons in Census block that 
speak a language other than English® 1.10 3.42 4.15 6.52 
First quartile 40 19.5 36.6 31.7 26.8 
Second quartile 70 26.1 39.1 31.9 36.2 
Third quartile 140 20.4 28.5 22.6 22.6 
Fourth quartile 190 20.8 29.2 21.9 21.4 


See notes at end of table. 
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Table D.5. Percentage of observed, occupied, residential, nonrespondent addresses with observed household attributes, 
by observed household attributes and selected characteristics: 2019—Continued 


Observed household attributes 


Privacy or security 


Presence of children concerns2 Outdoor living? Other outdoor decor+ 
Number of Chi-square Chi-square Chi-square Chi-square 
Selected characteristics addresses Percentage Statistic Percentage Statistic Percentage Statistic Percentage statistic 
Percent of persons in Census block without a 
high school diploma or the equivalent® 1.72 8.93* 1.20 5.26 
First quartile 100 19.0 23.2 25.3 27.4 
Second quartile 90 18.0 39.3 27.0 30.3 
Third quartile 90 24.5 38.3 26.6 26.6 
Fourth quartile 160 23.0 27.3 21.7 18.6 
Low Response Score? 5.19 12.37 * 7.40 21.10 * 
First quartile 100 22.8 41.6 32.7 25.7 
Second quartile 80 27.9 29.1 29.1 41.8 
Third quartile 90 24.2 37.4 22.0 26.4 
Fourth quartile 170 16.2 22.8 19.2 15.0 
Residential high-speed internet per 1000 
households’ 0.59 2.43 1.14 2.20 
600 or less 100 24.0 37.0 23.0 19.0 
601-800 160 21.3 27.7 22.6 26.5 
801 or more 180 20.1 31.0 27.2 26.1 — 
t Not applicable. 


+Reporting standards not met. There are too few cases for a reliable estimate. 


*p <0.05. 


1Addresses where children are thought to be present are those where observers identified indicators that suggest children live in the sampled unit. Examples include toys, bikes, car seats, strollers, outdoor swings/play sets, and 


child finder stickers for firefighters. 


2Addresses with privacy or security concerns are those where observers identified indicators that privacy or security is important to those living in the sampled unit. Examples include surveillance cameras, driveway gates, 
security company signs or stickers, and "No trespassing" signs. For multi-unit buildings, indicators that focused on the larger building in which the unit was located, such as entry buzzers or fences, were not included because they 


could be not directly associated with the sampled unit. 


3Addresses that value outdoor living are those where observers identified indicators that people in the sampled unit spend time outdoors. Examples include patio furniture, porch swings/benches, swing sets, sporting goods, and 


grills. This variable was added during data processing based on patterns observed in write-in responses. 


4Addresses with other outdoor decor are those where observers noted the presence of items used to decorate the exterior of the sampled unit. Examples include gnomes, flamingos, yard art, decorative flags, wreaths, Easter 

decorations, water features, or windchimes. This attribute focuses only on decorative items and does not include items that represent another household attribute collected during the observation (e.g., American flags were coded 
under patriotism only). This variable was added during data processing based on patterns observed in write-in responses. 
5These characteristics are based on variables available on the NHES:2019 sampling frame. 
6These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest prevalence of the characteristic in question and those that are in the fourth 


quartile are those with the highest prevalence. 


7The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial Census. A higher score corresponds to a lower expected mail return rate. 
This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 
8This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 
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NOTE: Percentages represent the percentage of addresses for which the attribute was observed. The household attributes shown here were not collected for addresses that could not be observed (e.g., cannot locate the address) or 
for which only a partial address observation could be completed (e.g., can observe the multi-unit building but cannot gain entry to observe the sampled unit). They also were not collected for addresses that were observed to be 
nonresidential or temporarily or permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from this 
analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but have not been changed to reflect sample size rounding. Details may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. Department of Commerce, American Community Survey (ACS), 2013-2017 and 
Decennial Census, 2010; and Federal Communications Commission (FCC), 2017. 
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Observed household attributes 


Privacy or security 


Appendix D. Additional Subgroup Analyses 


Presence of children concerns? Outdoor living? Other outdoor decor* 
Number of Chi-square Chi-square Chi-square Chi-square 
Selected observed characteristics addresses Percentage statistic Percentage statistic _ Percentage statistic Percentage statistic 
Total 440 21.4 t 31.2 t 24.6 t 24.6 t 
Presence of children! t 8.58 * 69.57 * 7.12 * 
Yes 90 t 43.6 57.4 35.1 
No or unclear5 350 t 278 15.7 21.7 
Privacy or security concerns2 8.58 * t 2.27 3.94 * 
Yes 140 29.9 t 29.2 30.7 
No 300 17.6 t 22.5 21.9 
Outdoor living? 69.57 * 2.27 t 20.11 * 
Yes 110 50.0 37.0 t 40.7 
No 330 12.1 29.3 t 19.3 
Patriotism® 2.62 0.85 5.46 * 66.27 * 
Yes 30 32.4 38.2 41.2 82.4 
No 410 20.5 30.6 23.2 19.8 
Welcoming decor’ 1.07 0.72 0.91 1.63 
Yes 30 13.8 24.1 17.2 34.5 
No 410 22.0 31.7 25.1 23.9 
Internet or television connectivity? 0.04 0.07 1.32 0.51 
Yes 30 20.0 33.3 33.3 30.0 
No 410 21.5 31.1 24.0 24.2 
Other outdoor decor* FAZ* 3.94 * 20.11 * t 
Yes 110 30.6 38.9 40.7 t 
No 330 18.4 28.7 19.3 t 
Structure type? 12.59 * 51.56 * 12.43 * 26.43 * 
Single-unit 260 26.9 43.2 29.5 33.0 
Attached 70 13.6 24.2 25.8 18.2 
Apartment 110 12.1 5.6 12.1 8.4 


See notes at end of table. 
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Table D.6. Percentage of observed, occupied, residential, nonrespondent addresses with observed household 
attributes, by observed household attributes and selected observed characteristics: 2019—Continued 


Observed household attributes 


Privacy or security 


_resence of children’ ___concerns*?___Outdoor living’ ___Other outdoor decor* _ 
Number of Chi-square Chi-square Chi-square Chi-square 
Selected observed characteristics addresses Percentage statistic Percentage statistic Percentage statistic Percentage statistic 
Mail access type? 13.10 * 29.73 * 14.50 * 8.84 * 
Mail slot or mailbox attached to the home 190 21.6 39.5 24.2 24.2 
Mailbox at the end of the driveway, across the 
street, or at the end of the road 160 28.7 34.4 33.1 29.9 
Mailbox, slot, or room in multi-unit building 50 6.5 $ 8.7 8.7 
No mailbox or slot in view 40 11.1 13.9 13.9 22.2 
Household income11 2.35 1.65 0.69 6.84 * 
Bottom third 140 22.1 27.9 25.7 17.6 
Middle third 230 19.3 34.3 23.6 26.6 
Top third 50 28.8 30.8 28.8 34.6 
tNot applicable. 
+Reporting standards not met. There are too few cases for a reliable estimate. 
*p < 0.05. 


1Addresses where children are thought to be present are those where observers identified indicators that suggest children live in the sampled unit. Examples include toys, bikes, car seats, strollers, outdoor swings/play sets, 
and child finder stickers for firefighters. 


2Addresses with privacy or security concerns are those where observers identified indicators that privacy or security is important to those living in the sampled unit. Examples include surveillance cameras, driveway gates, 
security company signs or stickers, and "No trespassing" signs. For multi-unit buildings, indicators that focused on the larger building in which the unit was located, such as entry buzzers or fences, were not included 
because they could be not directly associated with the sampled unit. 


3Addresses that value outdoor living are those where observers identified indicators that people in the sampled unit spend time outdoors. Examples include patio furniture, porch swings/benches, swing sets, sporting 
goods, and grills. This variable was added during data processing based on patterns observed in write-in responses. 


4Addresses with other outdoor decor are those where observers noted the presence of items used to decorate the exterior of the sampled unit. Examples include gnomes, flamingos, yard art, decorative flags, wreaths, Easter 
decorations, water features, or windchimes. This attribute focuses only on decorative items and does not include items that represent another household attribute collected during the observation (e.g., American flags were 
coded under patriotism only). This variable was added during data processing based on patterns observed in write-in responses. 


5This category includes both addresses where there was evidence suggesting children did not live there (e.g., the address was a senior living facility) and those where observers could not tell if children were living there. 
These have been combined due to the very small number of addresses where there was evidence suggesting children did not live there. 


6Patriotic addresses are those where observers identified indicators of American national or state-specific patriotism. They also include those with indicators of current or past involvement with the U.S military or pride in 
the U.S. military. 


7Addresses with welcoming decor are those where observers noted the presence of an object that welcomes visitors to the sampled unit, such as a welcome mat or welcome sign that explicitly says “welcome.” This variable 
was added during data processing based on patterns observed in write-in responses. 


8Addresses with internet or television connectivity are those where observers noted the presence of a satellite, cable, DirectTV dish or other electronic equipment, or a sign that indicates television or internet connectivity at 
the sampled unit. This variable was added during data processing based on patterns observed in write-in responses. 


%Attached structures include duplexes, townhouses, and rowhouses. Apartments include low-, medium-, and high-rise apartments. Structure type was not able to be determined for a very small number of cases, and these 
cases have been excluded from this analysis. 


104 few cases where mail was received in more than one way were categorized under "mail slot or mailbox attached to the home" since all the ways mail was received fit into that category. Mail access type was not able to 
be determined for a very small number of cases, and these cases have been excluded from this analysis. 


110bservers were asked to provide their best estimate of whether the address's household income was in the bottom third, middle third, or top third as compared to other households across the United States. 

NOTE: Percentages represent the percentage of addresses for which the attribute was observed. The household attributes shown here were not collected for addresses that could not be observed (e.g., cannot locate the 
address) or for which only a partial address observation could be completed (e.g., can observe the multi-unit building but cannot gain entry to observe the sampled unit). They also were not collected for addresses that were 
observed to be nonresidential or temporarily or permanently vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were 
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excluded from this analysis. Sample sizes are rounded to the nearest 10. Percentages are rounded to one decimal place but have not been changed to reflect sample size rounding. Details may not sum to totals because of 
rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Table D.7. Wald joint significance tests from binomial logistic regression using 
auxiliary variables to predict presence of children, privacy or security 
concerns, outdoor living, and other outdoor decor, by independent 
variables: 2019 


Privacy or 
Presence of security Outdoor Other outdoor 

Independent variables children concerns2 living3 decor* 
NHES:2019 bilingual screener mailings status 4.74 2.07 3.53 0.11 
Age of head of household® 8.58 3.52 1.67 10.25 
Gender of head of household> 1.84 2.64 4.83 0.48 
Education of head of household> 5.17 4.85 3.93 16.09 * 
Race of head of household® 10.21 * 1.34 2.00 8.52 
Household income® 3.64 2.50 4.91 7.27 
Whether household flagged as having children® 1.36 0.50 2.78 0.06 
Number of adults in household® 2.41 2.90 0.84 0.63 
Whether phone number available> 0.01 0.36 11.54 * 1.71 
Route type 0.00 8.40 * 1.56 0.00 
Dwelling type® 0.00 0.46 0.44 1.47 
Home tenure® 3.21 2.06 1.01 3.42 
Urbanicity>© 2.33 1.51 0.81 6.06 * 
Region> 5.32 9.33 * 3.47 5.44 
Race/ethnicity stratum? 4.71 4.23 1.08 0.64 
Tract poverty rate” 0.46 0.08 2.13 1.77 
Percent of households in Census block that include a 

child? 1.78 2.38 4.58 8.48 * 
Percent of persons in Census block that speak a language 

other than English? 1.38 0.45 3.00 5.67 
Percent of persons in Census block without a high school 

diploma or the equivalent’ 2.02 6.41 0.95 1.93 
Low Response Score® 1.08 5.97 2.21 8.91 * 
Residential high-speed internet per 1000 households? 2.59 7.77 * 0.95 0.88 
*p <.05. 


1Addresses where children are thought to be present are those where observers identified indicators that suggest children live in the sampled unit. 
Examples include toys, bikes, car seats, strollers, outdoor swings/play sets, and child finder stickers for firefighters. 

2Addresses with privacy or security concerns are those where observers identified indicators that privacy or security is important to those living in the 
sampled unit. Examples include surveillance cameras, driveway gates, security company signs or stickers, and "No trespassing" signs. For multi-unit 
buildings, indicators that focused on the larger building in which the unit was located, such as entry buzzers or fences, were not included because they 
could be not directly associated with the sampled unit. 
3Addresses that value outdoor living are those where observers identified indicators that people in the sampled unit spend time outdoors. Examples 
include patio furniture, porch swings/benches, swing sets, sporting goods, and grills. This variable was added during data processing based on patterns 
observed in write-in responses. 
4Addresses with other outdoor decor are those where observers noted the presence of items used to decorate the exterior of the sampled unit. Examples 
include gnomes, flamingos, yard art, decorative flags, wreaths, Easter decorations, water features, or windchimes. This attribute focuses only on decorative 
items and does not include items that represent another household attribute collected during the observation (e.g., American flags were coded under 
patriotism only). This variable was added during data processing based on patterns observed in write-in responses. 

5These variables are available on the NHES:2019 sampling frame. 

6This variable originally had four categories (urban, suburban, town, rural), but the town and rural categories had to be collapsed for this analysis due to 
sample size constraints. 

7These characteristics are based on American Community Survey (2013-2017) five-year estimates. 

8The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

°This variable is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The dependent variable for each model is a binary indicator of whether the attribute was observed at the address. Missing values on the 
independent variables are treated as a separate category. These household attributes were not collected for addresses that could not be observed (e.g., 
cannot locate the address) or for which only a partial address observation could be completed (e.g., can observe the multiunit building but cannot gain 
entry to observe the sampled unit). They also were not collected for addresses that were observed to be nonresidential or temporarily or permanently 
vacant. Addresses that ended up responding to NHES:2019 and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were 
excluded from this analysis. The pseudo R? value for the models are 0.20, 0.23, 0.14 and 0.24, respectively. The rounded sample size is 440. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 
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Table D.8. Wald joint significance tests from binomial logistic regression using 
observation variables to predict presence of children, privacy or 
security concerns, outdoor living, and other outdoor decor, by 
independent variables: 2019 


Privacy 

Presence of or security Outdoor Other outdoor 
Independent variables children concerns? living3 decor* 
Presence of children t 2.44 47.63 * 0.10 
Privacy or security concerns2 2.75 t 0.13 0.72 
Outdoor living? 47.26* 0.09 a 9.58 * 
Patriotism5 0.28 0.63 0.01 28.91 * 
Welcoming decor® 0.01 2.05 0.64 8.93 * 
Internet or television connectivity” 0.46 0.28 0.71 0.42 
Other outdoor decor* 0.16 0.69 9.06 * t 
Structure type® 3.42 19.51 * 0.92 7.95 * 
Mail access type? 1.32 8.24 * 3.84 0.42 
Household income! 1.92 1.57 0.74 0.50 
+Not applicable. 
*p <.05. 


1Addresses where children were thought to be present are those where observers identified indicators that suggest children live in the sampled unit. 
Examples include toys, bikes, car seats, strollers, outdoor swings/play sets, and child finder stickers for firefighters. 

2Addresses with privacy or security concerns are those where observers identified indicators that privacy or security is important to those living in the 
sampled unit. Examples include surveillance cameras, driveway gates, security company signs or stickers, and "No trespassing" signs. For multi-unit 
buildings, indicators that focused on the larger building in which the unit was located, such as entry buzzers or fences, were not included because they 
could be not directly associated with the sampled unit. 
3Addresses that value outdoor living are those where observers identified indicators that people in the sampled unit spend time outdoors. Examples 
include patio furniture, porch swings/benches, swing sets, sporting goods, and grills. This variable was added during data processing based on patterns 
observed in write-in responses. 
4Addresses with other outdoor decor are those where observers noted the presence of items used to decorate the exterior of the sampled unit. Examples 
include gnomes, flamingos, yard art, decorative flags, wreaths, Easter decorations, water features, or windchimes. This attribute focuses only on decorative 
items and does not include items that represent another household attribute collected during the observation (e.g., American flags were coded under 
patriotism only). This variable was added during data processing based on patterns observed in write-in responses. 

5Patriotic addresses are those where observers identified indicators of American national or state-specific patriotism. They also include those with 
indicators of current or past involvement with the U.S military or pride in the U.S. military. 

6Addresses with welcoming decor are those where observers noted the presence of an object that welcomes visitors to the sampled unit, such as a welcome 
mat or welcome sign that explicitly says “welcome.” This variable was added during data processing based on patterns observed in write-in responses. 
7Addresses with internet or television connectivity are those where observers noted the presence of a satellite, cable, DirectTV dish or other electronic 
equipment, or a sign that indicates television or internet connectivity at the sampled unit. This variable was added during data processing based on 
patterns observed in write-in responses. 

8Structure type categories are single-unit, attached and apartment. Attached structures include duplexes, townhouses, and rowhouses. Apartments include 
low-, medium-, and high-rise apartments. Structure type was not able to be determined for a very small number of cases, and these cases have been 
excluded from this analysis. 

9Mail access type categories are mail slot or mailbox attached to the home; mailbox at the end of the driveway, across the street or at the end of the road; 
mailbox, slot, or room in multi-unit building; and no mailbox or slot in view. A few cases where mail was received in more than one way were categorized 
under "mail slot or mailbox attached to the home" since all the ways mail was received fit into that category. Mail access type was not able to be determined 
for a very small number of cases, and these cases have been excluded from this analysis. 
10Household income categories are bottom third, middle third, or top third, as compared to other households across the United States. 
NOTE: The dependent variable for each model is a binary indicator of whether the attribute was observed at the address. Each attribute was excluded from 
the model predicting that attribute (e.g., the independent variable presence of children was excluded from the model predicting presence of children). 
These household attributes were not collected for addresses that could not be observed (e.g., cannot locate the address) or for which only a partial address 
observation could be completed (e.g., can observe the multiunit building but cannot gain entry to observe the sampled unit). They also were not collected 
for addresses that were observed to be nonresidential or temporarily or permanently vacant. Addresses that ended up responding to NHES:2019 and 
addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from this analysis. The pseudo R2 values for the 
models predicting presence of children, privacy or security concerns, outdoor living, and other outdoor decor are 0.18, 0.14, 0.17, and 0.21, respectively. 
The rounded sample size is 410. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 
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D.2.3. Agreement with Frame Variables 


For observations that were also available on the sampling frame, we calculated agreement 
rates between the observations and the frame variables (see section 7.3.1 for the overall 
results). In most cases, the frame variable that measured the same thing as the observation 
variable was a significant predictor of agreement; for example, the structure type agreement 
rate was significantly higher for multi-unit addresses than for single-unit ones. In the 
multivariate analyses, relatively few significant predictors of agreement were identified for 
most of the observation variables. Those that were identified varied across the four 
observation variables; no subgroup variable was a significant predictor of agreement for 
more than two of the four observation variables. Two variables were significant predictors 
of two agreement rates; for one of these (tract poverty rate) the direction of the relationship 
was consistent in both analyses, but for the other (head of household level of education) it 
was inconsistent. 


Occupancy status 


Only two of the 21 bivariate relationships were statistically significant (see table D.9 at the 
end of this section). To be consistent with the other analyses discussed in this appendix, we 
still conducted the multivariate analysis, which identified the following significant predictors 
of the structure type agreement rate (see table D.10 at the end of this section); however, the 
magnitude of the differences between subgroups was generally very small. 12 


e NHES:2019 bilingual mailing status: The agreement rate for address occupancy 
status was very similar by bilingual screener mailing status, where the agreement 
rate was about 95 percent for English-only mailings as well as for all bilingual 
mailings. 


e Gender of head of household: The agreement rate was somewhat lower for 
addresses missing frame data for this characteristic (93 percent) than it was for 
addresses with a male head of household (97 percent) or a female head of household 
(99 percent). 


e Dwelling type: The agreement rate was somewhat lower for single-unit structures 
(95 percent) than it was for multi-unit structures (99 percent). 


e Percentage of households in Census block that include a child: The agreement 
rate was lower for addresses in the third quartile (51st to 75th percentile) for this 
characteristic (90 percent) than it was for addresses in the other quartiles (98 
percent or above). 


12 For this model, the following predictors had to be dropped because one or more of the categories perfectly predicted the outcome: age 
of head of household, education of head of household, race/ethnicity of head of household, number of adults in household, and route 
type. Please interpret the results for this model with caution. 
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e Low Response Score: The agreement rate was slightly higher for the addresses with 
the lowest scores (i.e., those in the first quartile) (98 percent) than it was for those 
with the highest scores (i.e., those in the fourth quartile) (95 percent). 


e Residential high-speed internet per 1,000 households: The agreement rate was 
slightly higher for addresses in areas with higher rates of high-speed internet 
connectivity (98 percent in areas with 801 or more households per 1,000 with high- 
speed internet versus 93 percent in areas with 600 or less households per 1,000 with 
high-speed internet). 


Structure type 


For the subgroup analyses, 6 of the 21 bivariate relationships were statistically significant 
(see table D.9); the multivariate analysis identified two significant predictors of the structure 
type agreement rate (see table D.10).12 


e Region: The agreement rate was lower for addresses in the Northeast (78 percent) 
and West (84 percent) than it was for addresses in the South and Midwest (92 percent 
for both). 


e Tract poverty rate: The agreement rate was higher for addresses in high-poverty 
tracts (those with 20 percent or more of households in poverty) than it was for other 
addresses (92 percent versus 87 percent). 


Presence of children 


For the subgroup analyses, 10 of the 21 bivariate relationships were statistically significant 
(see table D.9); the multivariate analysis identified only one significant predictor of the 
presence of children agreement rate (see table D.10).14 


e Head of household level of education: The agreement rate for the presence of 
children was higher for addresses whose head of household had a graduate degree as 
their highest level of education (80 percent) than it was for addresses whose head of 
household had lower levels of education (65 percent or less in each group). It was 
also higher for addresses whose head of household was missing education 
information on the sampling frame (85 percent). 


Household income 


For the subgroup analyses, 10 the 21 bivariate relationships were statistically significant 
(see table D.9); the multivariate analysis identified three significant predictors of the 
household structure agreement rate (see table D.10).15 


13 For this model, the route type and dwelling type variables were excluded because they were the frame indicators of structure type, and 
the goal of this analysis was to determine whether other address characteristics affect the accuracy of these indicators on the frame. 

14 For this model, the presence of children variable was excluded because it was the frame indicator of the presence of children and the 
goal of this analysis was to determine whether other address characteristics affect the accuracy of this indicator on the frame. In addition, 
the number of adults in the household variable was dropped from the model due to multicollinearity. 

15 For this model, the household income variable was excluded from the model because it was the frame indicators of household income, 
and the goal of this analysis was to determine whether other address characteristics affect the accuracy of this indicator on the frame. 


D-34 


Appendix D. Additional Subgroup Analyses 
Head of household level of education: The agreement rate was higher for 
addresses whose head of household had a high school credential or less than a high 
school credential as their highest level of education (52 percent and 57 percent, 
respectively) than it was for addresses whose head of household had higher levels of 
education (36 percent or less in each group). It was also higher for addresses whose 
head of household was missing education information on the sampling frame (60 
percent). 


Home tenure: The agreement rate was lower for addresses inhabited by the 
homeowner than it was for addresses inhabited by renters (40 percent versus 
55 percent). 


Tract poverty rate: The agreement rate was higher for addresses in high-poverty 


tracts (those with 20 percent or more of households in poverty) than it was for other 
addresses (60 percent versus 38 percent). 
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Table D.9. Agreement rate between frame and observation variables for nonrespondent households, by variable 
and selected characteristics: 2019 


Occupancy status Structure type Presence of children Household income 
Agreement Chi-square Agreement Chi-square Agreement Chi-square Agreement Chi-square 
Selected characteristics rate statistic rate statistic rate statistic rate statistic 
Total 96.3 t 88.8 t 66.7 t 45.5 t 
NHES:2019 screener mailings language 0.56 1.19 4.32 3.53 
All bilingual mailings (English and Spanish) 95.5 87.2 72.6 42.3 
Mix of bilingual mailings and English-only 
mailings 97.0 89.2 62.7 49.8 
All English-only mailings 96.1 91.9 64.8 39.0 
Age of head of household! 4.65 1.98 33.20 * 4.72 
18-34 97.6 92.2 58.1 39.6 
35-44 95.2 88.2 478 46.0 
45-54 97.3 89.9 54.9 48.9 
55-64 96.8 90.7 68.3 37.8 
65 and older 100.0 85.0 78.9 54.0 
Missing 93.8 88.0 81.1 45.7 
Gender of head of household! 5.62 6.72 * 5.18 0.65 
Male 97.1 92.1 68.3 43.4 
Female 98.5 83.5 59.7 47.8 
Missing 93.2 90.2 71.9 45.4 
Education of head of household! 9.87 6.41 22.22 * 26.55 * 
Less than high school credential 96.8 86.7 65.4 57.4 
High school credential 90.3 90.4 59.4 51.8 
Some college 97.8 83.6 65.4 36.2 
Bachelor's degree 98.2 91.3 50.0 30.9 
Graduate degree 100.0 94.7 80.0 29.0 
Missing 96.6 91.4 85.3 59.6 
Race of head of household1 3.83 5.51 13.67* 15.62 * 
White, non-Hispanic 95.2 88.7 66.7 40.1 
Black, non-Hispanic 97.0 87.5 65.4 41.8 
Hispanic 98.0 85.1 56.8 57.9 
Other race, non-Hispanic 100.0 86.1 69.0 27.3 
Missing 93.7 94.2 83.1 52.6 


See notes at end of table 
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Table D.9. Agreement rate between frame and observation variables for nonrespondent households, by variable 
and selected characteristics: 2019—Continued 


Occupancy status Structure type Presence of children Household income 
Agreement Chi-square Agreement Chi-square Agreement Chi-square Agreement Chi-square 

Selected characteristics rate statistic rate statistic rate statistic rate statistic 

Household income! 2.38 9.10 19.12 * 31.95 * 
Less than $50,000 95.1 88.4 67.6 474 
$50,000-$74,999 98.3 81.5 53.3 60.0 
$75,000-$99,999 95.6 93.2 58.0 65.5 
$100,000 or higher 96.4 87.0 67.5 27.3 
Missing 100.0 96.8 94.4 T 

Household flagged as having children1 0.07 0.05 111.77 * 0.63 
Yes 96.7 88.2 29.7 42.7 
No 96.1 88.9 82.0 46.7 

Number of adults in household! 4.95 3.43 * 18.12 * 3.07 
1 adult 94.6 85.5 68.1 48.1 
2 adults 97.2 89.8 60.7 38.7 
3 adults or more 100.0 96.1 54.0 46.2 
Missing 100.0 96.8 94.4 # 

Phone number available! 0.54 3.21 2.64 0.09 
Yes 96.8 86.8 64.2 46.0 
No 95.3 91.8 71.9 44.5 

Route type? 4.32 * 24.59 * 11.41 * 1.50 
Street 95.3 83.6 62.3 43.6 
High rise 100.0 97.5 79.7 49.7 

Dwelling type1 2.95 27.82 * 13.45 * 1.78 
Single-unit 95.4 82.8 61.4 43.3 
Multi-unit 99.0 97.3 79.4 49.7 
Home tenure! 0.26 6.72 * 12.26 * 10.73 * 

Own 96.4 90.6 63.4 40.3 
Rent 96.6 83.9 65.2 55.0 
Missing 94.9 92.9 89.4 35.5 


See notes at end of table 
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Table D.9. Agreement rate between frame and observation variables for nonrespondent households, by variable 
and selected characteristics: 2019—Continued 


Occupancy status Structure type Presence of children Household income 
Agreement Chi-square Agreement Chi-square Agreement Chi-square Agreement Chi-square 

Selected characteristics rate statistic rate statistic rate statistic rate statistic 
Urbanicity 1.90 1.63 4.54 9.14 * 

Urban 94.9 89.8 72.2 49.5 

Suburban 97.3 87.5 62.8 42.2 

Town + + + + 

Rural t $ t $ 
Region! 4.46 14.98 * 0.73 1.91 

Northeast 98.4 77.9 64.6 45.2 

South 97.0 92.0 68.6 43.3 

Midwest 93.4 91.9 64.5 45.0 

West 98.3 83.8 67.7 52.9 
Race/ethnicity stratum2 0.24 1.56 3.36 8.27 * 

25% or more Black 95.5 90.0 68.1 46.2 

40% or more Hispanic 96.7 90.5 60.5 54.7 

Other 96.4 86.9 70.1 39.1 
Tract poverty rate2 0.89 2.30 0.17 21.21 * 

Less than 20% 96.9 87.3 66.1 38.1 

20% or more 94.9 91.7 68.1 60.4 

Percent of households in Census block that 

include a child2 15.44 * 3.00 15.30 * 5.73 

First quartile 98.8 92.3 77.8 36.4 

Second quartile 97.9 87.0 65.3 45.4 

Third quartile 89.8 86.1 72.8 49.1 

Fourth quartile 98.4 89.0 55.4 50.0 
Percent of persons in Census block that 

speak a language other than English2 5.76 1.05 4.58 4.31 

First quartile 94.7 91.3 61.0 45.5 

Second quartile 92.7 86.2 65.2 41.9 

Third quartile 99.2 89.8 73.7 40.3 

Fourth quartile 96.0 88.5 63.5 50.7 


See notes at end of table 
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Table D.9. Agreement rate between frame and observation variables for nonrespondent households, by variable 
and selected characteristics: 2019—Continued 


Occupancy status Structure type Presence of children Household income 


Agreement Chi-square Agreement Chi-square Agreement Chi-square Agreement Chi-square 
Selected characteristics rate statistic rate statistic rate statistic rate statistic 
Percent of persons in Census block without 


a high school diploma or the equivalent? 2.29 0.67 0.82 17.13 * 
First quartile 95.6 90.2 70.5 43.5 
Second quartile 96.5 87.5 66.3 35.2 
Third quartile 98.8 87.5 64.9 37.4 
Fourth quartile 95.1 89.2 65.8 57.8 
Low Response Score? 1.23 1.58 2.08 14.27 * 
First quartile 97.9 91.4 72.3 40.6 
Second quartile 96.3 85.7 67.1 33.3 
Third quartile 96.5 89.3 64.8 42.3 
Fourth quartile 95.1 88.9 64.1 55.4 
Residential high-speed internet per 1000 
households* 3.49 3.63 1.17 12.09 * 
600 or less 93.1 88.0 64.0 59.3 
601-800 96.5 92.0 65.2 41.6 
801 or more 97.7 86.2 69.6 40.4 _ 
tNot applicable. 


#Rounds to zero. 

+Reporting standards not met. There are too few cases for a reliable estimate. 

*p<0.05 

1These characteristics are based on variables available on the NHES:2019 sampling frame. 

2These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest prevalence of the characteristic in question and those that are 
in the fourth quartile are those with the highest prevalence. 

3The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial Census. A higher score corresponds to a lower expected 
mail return rate. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

4This characteristic is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The agreement rate is the percentage of addresses observed to have the same characteristic as found in the frame. Each analysis in this table is limited to cases that had data available from both sources. 
Rounded number of eligible cases is 540 for structure type, 410 for occupancy status, 440 for presence of children, and 480 for household income. Addresses that ended up responding to NHES:2019 and addresses 
that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from the analysis. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. Department of Commerce, American Community Survey (ACS), 
2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 2017. 
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Table D.10. Wald joint significance tests from binomial logistic regression 
predicting agreement rate between frame and observation variables 
for nonrespondent households, by independent variables: 2019 


Structure Occupancy Presence of Household 


Independent variables type! status2 children? income* 
NHES:2019 bilingual screener mailings status 0.86 6.66" 0.43 2.65 
Age of head of household® 3.63 t 10.66 6.61 
Gender of head of household® 2.74 8.69 * 2.69 1.69 
Education of head of households 8.91 t 11.73 * 11.53 * 
Race of head of household5 8.33 t 2.14 5.18 
Household income® 6.43 t 4.19 t 
Whether household flagged as having children® 0.00 0.82 i 0.58 
Number of adults in household5 3.83 t t 0.73 
Whether phone number available®> 2.07 0.00 0.20 0.28 
Route type> Tt tT 0.05 1.21 
Dwelling type® tT 6.18 * 2.68 0.77 
Home tenure® 1.93 0.58 0.36 7.40 * 
Urbanicity>® 2.01 3.41 1.92 0.74 
Regions 12.12 * 3.36 4.21 1.43 
Race/ethnicity stratum? 0.10 4.52 1.14 1.56 
Tract poverty rate” 5.49 * 2.17 0.40 4.40 * 
Percent of households in Census block that include a 

child” 4.25 10.84 * 7.37 2.16 
Percent of persons in Census block that speak a 

language other than English” 2.96 5.45 1.95 0.48 
Percent of persons in Census block without a high 

school diploma or the equivalent” 0.19 7.25 3.96 5.43 
Low Response Score® 2.78 8.44 * 7.21 1.91 
Residential high-speed internet per 1000 households? 5.31 6.69 * 1.70 1.58 
tNot applicable. 
*p <.05. 


1The independent variables route type and dwelling type were excluded from this model because they were used to create the dependent variable and 
were highly correlated with it. 
2The independent variables age of head of household, education of head of household, race of head of household, household income, number of adults in 
household and route type were dropped from this model because one or more of the categories perfectly predicted the dependent variable. 

3The independent variable that flagged some households as having children was excluded from this model because it was used to create the dependent 
variable and was highly correlated with it. The independent variable number of adults in household was dropped from this model due to multicollinearity. 
4The independent variable household income was excluded from this model because it was used to create the dependent variable and was highly 
correlated with it. 

5These variables are available on the NHES:2019 sampling frame. 
6This variable originally had four categories (urban, suburban, town, rural), but the town and rural categories had to be collapsed for this analysis due to 
sample size constraints. 

7 These characteristics are based on American Community Survey (2013-2017) five-year estimates. 

8The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial 
Census. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

°This variable is based on tract-level estimates of Internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: Missing values on the independent variables are treated as a separate category. The dependent variable for each model is a binary indicator of 
whether an address was observed to have the same characteristic as found in the NHES:2019 sampling frame. For each model, addresses that were missing 
data for either of the variables used to create the dependent variable were excluded from that analysis. Addresses that ended up responding to NHES:2019 
and addresses that had at least one undeliverable as addressed (UAA) NHES:2019 mailing were excluded from the analysis. The pseudo R2 values for the 
models predicting agreement for structure type, occupancy status, presence of children, and household income are 0.21, 0.53, 0.17, and 0.12, respectively. 
The rounded sample size is 540 for structure type, 400 for occupancy status, 440 for presence of children, and 480 for household income. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. 
Department of Commerce, American Community Survey (ACS), 2013-2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 
2017. 
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This appendix provides more information about the effort to recruit the 400 addresses 
sampled for the interview component of the study. Three modes of recruitment were used— 
mailed materials, outbound phone calls, and in-person visits. As noted in chapter 2, all 
households in the interview component sample were sent an invitation letter and two 
reminder postcards. In addition, two rounds of outbound phone calls were made to the 270 
addresses for which a phone number was available on the NHES:2019 sampling frame. 
Finally, three weeklong waves of in-person recruitment were conducted. Follow-up 
continued for each address until an interview had been scheduled or conducted or a hard 
refusal had been received. 


Eighty-five interviews were ultimately conducted. Nearly 80 percent of the 85 interviews 
were recruited through in-person visits, about 15 percent through mailed contacts, and 
about 5 percent through outbound calls. In the following sections, we provide a summary of 
the outcomes of the phone and in-person recruitment activities. The mailed recruitment 
activities are not discussed in detail in this appendix because the main outcome of interest, 
the undeliverable as addressed (UAA) outcome, was discussed in chapter 7 (see section 7.2). 


E.1 Outbound Phone Calls 


During each of the two rounds of outbound phone calls, staff aimed to call each address up 
to two times; about half of the addresses (46 percent) ended up being called four times. The 
total number of phone calls made to each address ultimately ranged from 1 to 6. About half 
of the addresses were called less than four times; 9 percent received one call, 34 percent 
received two calls, and 9 percent received three calls. Addresses were called less than four 
times if, for example, an interview was conducted or a hard refusal was received prior to the 
second wave of phone calls. In some cases, phone calls were stopped for an address because 
information gathered during the initial calls suggested that number was not a good way to 
reach the sampled address (e.g., the number was not in service or the person that answered 
indicated that the phone number was not associated with the sampled address). A small 
number of addresses (2 percent) received 5 or more calls. This occurred, for example, if the 
address requested additional callbacks at a different time. 


Based on these calling efforts, we determined the final call outcome for each address. The 
final call outcome is based on the best or most informative call outcome. For example, if an 
address was called three times, and the first two calls were not answered and the last call 
resulted in a scheduled interview, then the final (and best) outcome is the one from the last 
call. As another example, if the first two calls went unanswered but the third resulted in a 
hard refusal, then the final (and most informative) outcome is the hard refusal. 


Table E.1 presents the distribution of the final call outcomes. An interview being scheduled 
was the final call outcome for 1 percent of the called addresses; the phone calls were the 
successful mode of recruitment for 5 of the conducted interviews. Three of these were 
scheduled during the outbound calls, and the other two were scheduled as a result of the 
participants calling the team to schedule an interview. Two additional interview participants 
also mentioned during the interview that they had received the recruitment calls. 
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However, the most common final call outcome was for there to be no answer. For about a 
third of the called addresses, the final call outcome was leaving a voicemail (34 percent). For 
another third, the final call outcome was no answer (without the possibility of leaving a 
voicemail message) (34 percent). For a few additional addresses, the final call outcome was 
a busy signal (3 percent). 


Table E.1. Number and percentage distribution of final call outcome for addresses 
sampled for interview component: 2019 


Number of Percentage of 

Final call outcome addresses addresses 
Total 273 100.0 
Interview scheduled 3 1.1 
Soft refusal 8 2.9 
Hard refusal 8 2.9 
Hang up during introduction 13 4.8 
Voicemail left 92 33.7 
No answer, voicemail not possible 93 34.1 
Busy signal 8 2.9 
Language barrier 1 0.4 
Ineligible person 2 0.7 
Number not in service 25 9.2 
Number not associated with sampled address 16 5.9 
Not a household number 3 1.1 
Other 1 0.4 


NOTE: The final call outcome is the most informative outcome of all calls that were made to an address. Details may not sum to totals because of rounding. 
Sample sizes have not been rounded. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 


For about one in six called addresses, the final call outcome was the determination that the 
number was either not in service (9 percent) or not associated with the sampled address (6 
percent not associated with the sampled address, 1 percent not a household number). For 
example, in some cases, the person who answered the phone said he or she used to live at 
the sampled address but no longer did so. 


For about one in nine called addresses, the final outcome was a refusal (3 percent hard 
refusal, 3 percent soft refusal, 5 percent hang up during introduction). These households 
typically did not provide a specific reason why they did not want to participate in the study. 
They still received in-person visits due to the possibility that the person who refused did not 
actually live at the sampled address (e.g., if the number on the sampling frame was not 
associated with the sampled address). Finally, for about 1 percent of addresses, the final call 
outcome was either a language barrier or an ineligible household member (e.g., not 
responsible for checking the mail). 


We also conducted a subgroup analysis to determine whether certain final phone outcomes 
were more or less common for particular subgroups (see table E.2). For this analysis, we 
collapsed the final phone outcomes into American Association for Public Opinion Research 
(AAPOR) standard disposition codes: interview scheduled (1 percent); eligible, no interview 
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scheduled (11 percent); unknown eligibility (72 percent); and ineligible (16 percent).16 
Because very few addresses had an interview scheduled as a result of the calls, there were 
no subgroups for which this was a very common outcome. There were, however, some 
patterns in terms of the characteristics of the addresses that ended up having an interview 
scheduled as a result of the calls—they all had a Black or White head of household who was 
age 55 or older (or missing age information) and was female (or missing gender 
information), they were all single-adult households, and they were all owner inhabited. 
There also were not very many notable subgroup differences in the likelihood of eligible, no 
interview or ineligible outcomes. However, there were a few subgroups that were somewhat 
more likely than average to end up with these outcomes. Addresses that were rented (23 
percent) and those in the Black oversample stratum (22 percent) were somewhat more 
likely than average to have an eligible, no interview (refusal) outcome. Addresses with a 
Black head of household (34 percent), whose head of household had completed some college 
but did not have a degree (27 percent), and that were missing age information for the head 
of household (26 percent) were somewhat more likely than average to have an ineligible 
outcome. 


16 The final phone outcomes were categorized into standard AAPOR disposition codes as follows: (1) Interview scheduled includes 
addresses where the final call outcome was a scheduled interview; (2) Eligible, non-interview includes addresses where the final call 
outcome was a soft or hard refusal, hang up during introduction, or other; (3) Unknown eligibility includes addresses where the final call 
outcome was voicemail left, language barrier, busy signal, no answer/voicemail not possible, and ineligible persons; and (4) /neligible 
includes addresses where the final call outcome was number not in service, number not associated with sampled address, and not a 
household number. 
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Table E.2. Number of addresses and percentage distribution of final call outcome for addresses sampled for interview 
component, by selected characteristics: 2019 


Final call outcome 


Number Interview Eligible, no Unknown 
Selected characteristics of addresses scheduled interview scheduled eligibility Ineligible 
Total 273 1.1 11.0 71.8 16.1 
Age of head of household1 
18-34 23 0.0 8.7 69.6 21.7 
35-44 43 0.0 7.0 79.1 14.0 
45-54 49 0.0 18.4 69.4 12.2 
55-64 52 1.9 1.9 80.8 15.4 
65 and older 41 24 19.5 73.2 4.9 
Missing 65 L: 10.8 61.5 26.2 
Gender of head of household! 
Male 86 0.0 10.5 79.1 10.5 
Female 91 1.1 12.1 70.3 16.5 
Missing 96 2.1 10.4 66.7 20.8 
Education of head of household1 
Less than high school credential 79 1.3 12.7 69.6 16.5 
High school credential 54 1.9 14.8 74.1 9.3 
Some college 51 0.0 9.8 62.8 27.5 
Bachelor's degree 40 2.5 12.5 82.5 2.5 
Graduate degree 15 0.0 0.0 80.0 20.0 
Missing 34 0.0 5.9 70.6 23.5 
Race of head of household! 
White, non-Hispanic 110 1.8 8.2 79.1 10.9 
Black, non-Hispanic 35 2.9 14.3 48.6 34.3 
Hispanic 75 0.0 14.7 72.0 13.3 
Other race, non-Hispanic 17 0.0 11.8 76.5 11.8 
Missing 36 0.0 8.3 69.4 22.2 
Household income1 
Less than $50,000 119 0.8 16.8 66.4 16.0 
$50,000-$74,999 38 0.0 10.5 86.8 2.6 
$75,000-$99,999 33 0.0 0.0 78.8 212. 
$100,000 or higher 71 28 7.0 69.0 21.1 
Missing 12 0.0 8.3 75.0 16.7 


See notes at end of table. 
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Table E.2. Number of addresses and percentage distribution of final call outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Final call outcome 


Number Interview Eligible, no Unknown 

Selected characteristics of addresses scheduled interview scheduled eligibility Ineligible 
Household flagged as having children1 

Yes 100 1.0 7.0 78.0 14.0 

No 173 1.2 13.3 68.2 17.3 
Number of adults in household1 

1 adult 150 2.0 12.0 68.0 18.0 

2 adults 79 0.0 12.7 73.4 13.9 

3 adults or more 32 0.0 3.1 84.4 12.5 

Missing 12 0.0 8.3 75.0 16.7 
Route type? 

Street 213 0.9 8.5 75.6 15.0 

High rise 60 1.7 20.0 58.3 20.0 
Dwelling type1 

Single-unit 209 1.0 8.6 77.0 13.4 

Multi-unit 64 1.6 18.8 54.7 25.0 
Home tenure! 

Own 163 1.8 4.9 80.4 12.9 

Rent 90 0.0 23.3 54.4 22.2 

Missing 20 0.0 5.0 80.0 15.0 
Urbanicity1 

Urban 99 2.0 10.1 68.7 19.2 

Suburban 161 0.6 12.4 72.7 14.3 

Town 2 0.0 0.0 100.0 0.0 

Rural 11 0.0 0.0 81.8 18.2 
Region! 

Northeast 75 0.0 13.3 69.3 17.3 

South 60 1.7 6.7 73.3 18.3 

Midwest 70 2.9 8.6 72.9 15.7 

West 68 0.0 14.7 72.1 13.2 


See notes at end of table. 
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Table E.2. Number of addresses and percentage distribution of final call outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Final call outcome 


Number Interview Eligible, no Unknown 

Selected characteristics of addresses scheduled interview scheduled eligibility Ineligible 
Race/ethnicity stratum2 

25% or more Black 41 2.4 22.0 53.7 22.0 

40% or more Hispanic 85 0.0 11.8 75.3 12.9 

Other 147 1.4 7.5 74.8 16.3 
Tract poverty rate2 

Less than 20% 176 1.1 9.7 75.6 13.6 

20% or more 97 1.0 13.4 65.0 20.6 
Percent of households in Census block that include a 

child2 

First quartile 59 3.4 11.9 59.3 25.4 

Second quartile 67 0.0 13.4 76.1 10.5 

Third quartile 54 1.9 5.6 83.3 9.3 

Fourth quartile 93 0.0 11.8 69.9 18.3 
Percent of persons in Census block that speak a 

language other than English2 

First quartile 29 3.5 3.5 75.9 17.2 

Second quartile 58 0.0 12.1 70.7 17.2 

Third quartile 72 2.8 12.5 72.2 12.5 

Fourth quartile 114 0.0 11.4 71.1 17.5 
Percent of persons in Census block without a high 

school diploma or the equivalent? 

First quartile 62 1.6 4.8 80.7 12.9 

Second quartile 59 1.7 11.9 62.7 23.7 

Third quartile 56 1.8 14.3 69.6 14.3 

Fourth quartile 96 0.0 12.5 72.9 14.6 
Low Response Score? 

First quartile 65 0.0 4.6 80.0 15.4 

Second quartile 55 3.6 10.9 67.3 18.2 

Third quartile 58 1.7 6.9 79.3 12.1 

Fourth quartile 95 0.0 17.9 64.2 17.9 


See notes at end of table. 
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Table E.2. Number of addresses and percentage distribution of final call outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Final call outcome 


Eligible, 
Number Interview no interview Unknown 
Selected characteristics of addresses scheduled scheduled eligibility Ineligible 
Residential high-speed internet per 1000 households* 
600 or less 73 1.4 13.7 63.0 21.9 
601-800 101 1.0 12.9 73.3 12.9 
801 or more 99 1.0 7.1 76.8 15.2 


1These characteristics are based on variables available on the NHES:2019 sampling frame. 

2These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest prevalence of the characteristic in question and those that are in the fourth 
quartile are those with the highest prevalence. 
3The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial Census. A higher score corresponds to a lower expected mail return rate. 
This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

4This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The final call outcome for each case was categorized into American Association for Public Opinion Research (AAPOR) standard disposition outcomes. The categorization was conducted as follows: (1) Interview scheduled 
includes cases where the recruitment call resulted in a scheduled interview. (2) Eligible, non-interview includes cases where there was a soft or hard refusal, hang up during introduction, or "other" outcome. (3) Unknown eligibility 
includes voicemail left, language barrier, busy signal, no answer/voicemail not possible, and ineligible person. (4) /neligible includes number not in service, number not associated with sampled address, and not a household 
number. Details may not sum to totals because of rounding. Sample sizes have not been rounded. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. Department of Commerce, American Community Survey (ACS), 2013-2017 and 
Decennial Census, 2010; and Federal Communications Commission (FCC), 2017. 
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E.2. In-Person Visits 


Field staff made three weeklong in-person recruitment trips to each site. Almost all (98 
percent) of the 400 addresses sampled for the interview component of the study were visited 
at least once. Addresses were visited up to five times; more specifically, about a third were 
visited once (34 percent); just under two-fifths were visited twice (38 percent); and just over 
a quarter were visited three or more times (19 percent were visited three times and 7 
percent were visited 4 or more times). The remaining 2 percent were not visited either 
because the household had already refused participation in response to the earlier contacts 
or because the results of the observation component of the study had indicated it was 
impossible to reach the address or the address was vacant. 


Table E.3 presents the distribution of the final in-person visit outcome resulting from these 
visits. Similar to the final call outcome, the final visit outcome is the best or most informative 
outcome across all the visits made to a household. Catching people at home presented one of 
the biggest challenges for recruitment. One of the most common final visit outcomes was not 
finding people at home during the visits (22 percent). When interview staff did not find 
anyone at home, they left a Sorry We Missed You card. Interviewers mentioned that a few 
interview participants later noted that incorporating a “human element” into the cards (such 
as leaving a handwritten note or personal phone number on the card or leaving a personal 
business card with the card) had made them more open to completing the interview. 


Table E.3. Number and percentage distribution of final in-person visit outcome for 
addresses sampled for interview component: 2019 


Percentage of 


Final in-person visit outcome Number of addresses addresses 

Total 400 100.0 
Interview completed 85 21.3 
Short interview completed 6 1.5 
Soft refusal 38 9.5 
Hard refusal 67 16.8 
No answer (evidence someone is home) 19 4.8 
No answer (no evidence someone is home) 89 22.3 
Physical or mental disability 2 0.5 
Language barrier 11 2.8 
Unable to reach household or unsafe 38 9.5 
Ineligible person 24 6.0 
Ineligible household 2 0.5 
Vacant or non-existent unit! 10 2.5 
Not visited due to prior recruitment or observation outcome2 8 2.0 
Other 1 0.3 


1These addresses were not determined to be vacant during the observations but were determined to be vacant during the in-person recruitment visits. It 
is possible the address's vacancy status changed between the observation and the in-person visit. It is also possible that the field staff received new 
information during the in-person visit that had not been available during the observation (e.g., gaining entry to a multi-unit building that had been 
restricted during the observation). 

2These addresses did not receive in-person recruitment visits because they were determined to be vacant, were unable to be reached, or had provided a 
hard refusal before the in-person recruitment period (either as part of the address observations or in response to the mail-based interview recruitment). 
NOTE: The final visit outcome is the most informative outcome of all visits that were made to an address. Details may not sum to totals because of 
rounding. Sample sizes have not been rounded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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Making repeated visits to the addresses helped to ensure contact with someone at the 
selected address for many of the other addresses. In general, the best times to reach sample 
members at home varied somewhat across sites —while at one site weekends seemed to be 
when more people were home, at the other three sites, weeknight evenings were the most 
successful time to find people home. For 85 addresses (21 percent), the final in-person 
outcome was that an interview was completed; for another 2 percent, it was that a short 
interview was completed. For about a third of the addresses, the person who was reached 
declined to participate, whether actively or passively; about 17 percent of addresses resulted 
in a hard refusal, about 10 percent in a soft refusal, and about 5 percent did not answer the 
door even though it was clear that someone was home. For about 9 percent of the addresses, 
the person who was reached was unable to participate in the interview because of a language 
barrier (3 percent), a physical or mental disability (1 percent), or because the person did not 
meet the eligibility requirements (6 percent). 


When finding household members at home, two types of reasons for refusal were commonly 
provided. First, several household members simply stated they did not want to participate 
or were not interested—and made it clear that their opinion was not going to change (e.g., 
asking the team not to come back, closing the door while the team was talking). Second, 
several household members indicated that they were too busy to participate (e.g., taking care 
of small children; heading out soon for an appointment, work, or school pickup). In some 
cases, these household members expressed openness to participating later, but in others, 
they said they were too busy to schedule for a later date. 


In a smaller number of cases, doorstep conversations were a critical factor in convincing 
household members to participate in the interview. Interviewers noted that, in a few cases, 
mentioning the Department of Education increased household members’ openness to or 
interest in participating in the study; this seemed to resonate more than mentioning the 
National Household Education Survey or the National Center for Education Statistics. In a 
few cases, when interviewers mentioned they were visiting “from out of town,” this seemed 
to cause household members to become less interested in participating. In addition, across 
all sites, interviewers encountered sample members and neighbors who were suspicious of 
them or were concerned about the interview request being a scam. Some households 
watched the team approach and reviewed the badges they were wearing to verify their 
identity. Even when the household members did not think the interview was a scam, some 
of them were concerned with “the government” coming to their address and collecting 
information about them. Such fears included concerns that the interview staff were part of 
Child Protective Services or Immigration Services. In a few cases, household members 
refused to participate in the interview because they were upset about having seen the team 
members in the neighborhood before, while making previous recruitment visits or working 
on the household observation. 


Even when addresses were visited repeatedly, it was impossible to reach about 10 percent 
of the households because they were inaccessible (e.g., because of gatekeepers or locked 
buildings) or unsafe to reach (e.g., because of guard dogs). Finally, for about 3 percent of 
addresses, the final in-person visit outcome suggested ineligibility to participate in an 
interview —either because none of the household members met the interview eligibility 
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requirements (e.g., they had only just moved to the address) (1 percent) or because the 
address appeared to be vacant or non-existent (3 percent). 


We also conducted a subgroup analysis to determine whether certain final in-person visit 
outcomes were more or less common for particular subgroups (see table E.4). For this 
analysis, we again collapsed the final outcomes into AAPOR standard disposition codes: 
interview conducted (21 percent); eligible, no interview scheduled (33 percent); unknown 
eligibility (41 percent); and ineligible (5 percent).!”7 Addresses in the Black oversample 
stratum were somewhat more likely than average to have an interviewed-conducted 
outcome (36 percent). Addresses where the head of household’s educational attainment was 
high school and those in rural areas were more likely than average to have an eligible, no 
interview scheduled (refusal) outcome (44 percent and 50 percent, respectively). 
Interestingly, addresses whose Census Low Response Score was in the lowest quartile (i.e., 
expected to be the most responsive) were more likely to have an eligible, no interview 
scheduled outcome than cases whose score was in the highest quartile (i.e., expected to be 
the least responsive) (45 percent versus 27 percent). Addresses that were missing frame 
information were more likely than average to have an unknown eligibility outcome (e.g., 57 
percent of addresses missing information about the number of adults in the household). 
Although the ineligible outcome tended to be uncommon for most subgroups, there were a 
few subgroups where it occurred more often than average: those where home tenure was 
missing (14 percent) and those with the lowest percentage of persons in the Census block 
that speak a language other than English (12 percent). 


17 The final in-person visit outcomes were categorized into standard AAPOR disposition codes as follows: (1) Interview conducted 
includes addresses where the final visit outcome was a completed interview; (2) Eligible, non-interview includes addresses where the 
final visit outcome was short interview completed, no answer (evidence someone is home), soft refusal, hard refusal, or other; (3) 
Unknown eligibility includes addresses where the final visit outcome was physical or mental disability, language barrier, unable to reach 
household or unsafe, no answer (no evidence someone is one home), or ineligible person; and (4) Ineligible includes addresses where the 
final visit outcome was ineligible household, vacant or non-existent unit, or addresses not visited due to observation outcome. 
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Table E.4. Number of addresses and percentage distribution of final visit outcome for addresses sampled for interview 
component, by selected characteristics: 2019 


Final visit outcome 


Number Interview Eligible Unknown 
Selected characteristics of addresses conducted non-interview eligibility Ineligible 
Total 400 21.3 32.8 41.0 5.0 
Age of head of household1 
18-34 38 21.1 31.6 44.7 2.6 
35-44 63 27.0 31.8 39.7 1.6 
45-54 59 18.6 42.4 37.3 1.7 
55-64 61 26.2 32.8 32.8 8.2 
65 and older 54 18.5 37.0 37.0 74 
Missing 125 18.4 27.2 48.0 6.4 
Gender of head of household1 
Male 129 20.9 30.2 43.4 5.4 
Female 125 20.8 36.8 38.4 4.0 
Missing 146 21.9 31.5 41.1 5.5 
Education of head of household1 
Less than high school credential 120 23.3 29.2 40.8 6.7 
High school credential 71 25.4 43.7 25.4 5.6 
Some college 71 25.4 31.0 42.3 1.4 
Bachelor's degree 50 18.0 34.0 44.0 4.0 
Graduate degree 22 22.7 36.4 40.9 0.0 
Missing 66 10.6 27.3 54.6 7.6 
Race of head of household1 
White, non-Hispanic 145 22.8 36.6 36.6 41 
Black, non-Hispanic 55 29.1 21.8 41.8 7.3 
Hispanic 105 21.0 39.1 35.2 48 
Other race, non-Hispanic 25 20.0 24.0 52.0 4.0 
Missing 70 12.9 27.1 54.3 5.7 


See notes at end of table. 
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Table E.4. Number of addresses and percentage distribution of final visit outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Selected characteristics 

Household income! 
Less than $50,000 
$50,000-$74,999 
$75,000-$99,999 
$100,000 or higher 
Missing 

Household flagged as having children1 
Yes 
No 

Number of adults in household! 
1 adult 
2 adults 
3 adults or more 
Missing 

Phone number available! 
Yes 
No 

Route type? 
Street 
High rise 

Dwelling type1 
Single-unit 
Multi-unit 

Home tenure! 
Own 
Rent 
Missing 


See notes at end of table. 


Number 
of addresses 


192 
46 
42 
85 
35 


120 
280 


233 
99 
33 
35 


273 
127 


274 
126 


259 
141 


200 
149 
51 


Interview 
conducted 


21.9 
26.1 
26.2 
17.7 
14.3 


21.7 
21.1 


22.8 
17.2 
30.3 
14.3 


24.2 
15.0 


20.4 
23.0 


20.9 
22.0 


18.5 
28.2 
11.8 


Final visit outcome 


Eligible 
non-interview 


28.1 
32.6 
42.9 
42.4 
22.9 


31.7 
33.2 


30.5 
39.4 
39.4 
22.9 


34.4 
29.1 


36.9 
23.8 


37.5 
24.1 


41.5 
25.5 
19.6 


Unknown 
eligibility 


44.3 
34.8 
31.0 
35.3 
57.1 


42.5 
40.4 


41.2 
39.4 
27.3 
57.1 


37.4 
48.8 


37.2 
49.2 


36.3 
49.7 


37.5 
40.9 
54.9 


Ineligible 
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5.7 
6.5 
0.0 
4.7 
5.7 


4.2 
5.4 


5.6 
4.0 
3.0 
5.2 


4.0 
7.1 


5.5 
4.0 


5.4 
4.3 


2.5 
5.4 
13.7 
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Table E.4. Number of addresses and percentage distribution of final visit outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Selected characteristics 


Urbanicity! 
Urban 
Suburban 
Town 
Rural 

Region! 
Northeast 
South 
Midwest 
West 

Race/ethnicity stratum2 


25% or more Black 
40% or more Hispanic 
Other 
Tract poverty rate2 
Less than 20% 
20% or more 
Percent of households in Census block that include a child? 


First quartile 

Second quartile 

Third quartile 

Fourth quartile 

Percent of persons in Census block that speak a language 
other than English2 

First quartile 

Second quartile 

Third quartile 

Fourth quartile 


Number 
of addresses 


165 
216 
3 
16 


100 
100 
100 
100 


73 
127 
200 


248 
152 


93 
87 
99 
121 


41 
78 
107 
174 


Interview 
conducted 


24.2 
20.4 
0.0 
6.3 


20.0 
23.0 
28.0 
14.0 


35.6 
19.7 
17.0 


18.2 
26.3 


19.4 
26.4 
15.2 
24.0 


29.3 
21.8 
19.6 
20.1 


Final visit outcome 


Eligible 
non-interview 


29.1 
34.7 

0.0 
50.0 


37.0 
34.0 
21.0 
39.0 


15.1 
37.8 
36.0 


38.3 
23.7 


35.5 
36.8 
30.3 
29.8 


24.4 
25.6 
40.2 
33.3 


Unknown 
eligibility 


42.4 
38.9 
100.0 
43.8 


37.0 
40.0 
45.0 
42.0 


41.1 
40.2 
41.5 


37.9 
46.1 


41.9 
35.6 
43.4 
42.2 


34.2 
51.3 
34.6 
42.0 


Ineligible 


4.2 
6.0 
0.0 
0.0 


6.0 
3.0 
6.0 
5.0 


8.2 
2.4 
5.5 


5.7 
4.0 


3.2 
1.2 
11.1 
4.1 


12.2 
1.3 
5.6 
4.6 


See notes at end of table. 
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Table E.4. Number of addresses and percentage distribution of final visit outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Final visit outcome 


Number Interview Eligible Unknown 
Percent of persons in Census block without a high school 
diploma or the equivalent2 
First quartile 82 14.6 36.6 43.9 4.9 
Second quartile 78 20.5 29.5 43.6 6.4 
Third quartile 81 22.2 35.8 38.3 3.7 
Fourth quartile 159 24.5 30.8 39.6 5.0 
Low Response Score? 
First quartile 86 15.1 45.4 33.7 5.8 
Second quartile 70 15.7 34.3 45.7 4.3 
Third quartile 78 30.8 29.5 35.9 3.9 
Fourth quartile 165 22.4 27.3 44.9 5.5 
Residential high-speed internet per 1000 households* 
600 or less 116 25.9 24.1 45.7 4.3 
601-800 150 21.3 34.7 38.0 6.0 
801 or more 134 17.2 38.1 40.3 4.5 


1These characteristics are based on variables available on the NHES:2019 sampling frame. 

2These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest prevalence of the characteristic in question and those that are in the 
fourth quartile are those with the highest prevalence. 

3The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial Census. A higher score corresponds to a lower expected mail 
return rate. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

4This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The final visit outcome for each case was categorized into American Association for Public Opinion Research (AAPOR) standard disposition outcomes. The categorization was conducted as follows: (1) Interview 
conducted includes cases where the visit resulted in a completed interview. (2) Eligible, non-interview includes short interview completed, no answer (evidence someone is home), soft refusal, hard refusal, and other. (3) 
Unknown eligibility includes physical or mental disability, language barrier, unable to reach household or unsafe, no one home, and ineligible person. (4) /neligible includes ineligible household, vacant or non-existent unit, 
and addresses not visited due to observation outcome. Details may not sum to totals because of rounding. Sample sizes have not been rounded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. Department of Commerce, American Community Survey (ACS), 2013- 
2017 and Decennial Census, 2010; and Federal Communications Commission (FCC), 2017. 
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E.3. Help Desk 


A help desk phone number and email address were provided in the mailings and the Sorry 
We Missed You cards that were left at addresses. When outbound calls were made to 
addresses and a voicemail was left, the help desk phone number was also included in the 
voicemail message. Thirty inbound calls and three e-mails were received by the help desk as 
aresult. The recruitment mailings were the communication mode that resulted in the largest 
number of calls (19 out of 30) to the help desk. When calling or e-mailing the helpdesk, most 
household members were trying to schedule an interview or confirm that they were eligible 
to participate in the study. They tended to note that they were responding to the 
“Department of Education” communication they had received. A very small number of calls 
were to indicate that the sampled address was not interested in participating in the study. 


E.4 Final Study Outcomes 


Based on the combined outcomes across all recruitment activities —that is mail, outbound 
calls, and in-person visits, and help desk contacts —each household was assigned a final 
study outcome based on the best or most informative outcome. For example, if an address 
called the help desk in response to one of the mailings to ask that it not be visited, then the 
final study outcome is a refusal. Or if the outbound calls were not answered but the in-person 
visits resulted in a language barrier, then the final study outcome is language barrier. Table 
E.5 shows the distribution of the final study outcomes. The distribution closely resembles 
the distribution of the final in-person visit outcome (since, for most cases, that was the most 
informative recruitment activity). For example, for about a fifth of the sample (21 percent), 
the final study outcome was a completed interview. For about a third (31 percent), it was a 
refusal. For about a quarter (24 percent), there was no answer (e.g., the interviewer knocked 
on the door or rang the doorbell, but no one answered), and for about 10 percent, the address 
could not be reached due to access controls or safety issues. 


Similar to the categorization for the outbound phone and visit outcomes, each of the final 
outcomes was also grouped into AAPOR’s standard disposition codes.!8 Subgroup analysis 
indicated similar patterns as those seen for the phone and visit outcome analyses (see table 
E.6). For example, addresses in the Black oversample stratum were more likely than average 
to participate in an interview (36 percent) and less likely to have an eligible, no interview 
final study outcome (16 percent). Rural addresses were less likely than average to 
participate in an interview (6 percent) and more likely to have an eligible, no interview study 
outcome (44 percent). Addresses that were missing frame information were more likely than 
average to have an unknown eligibility final study outcome; for example, 60 percent of 
addresses that were missing household income information had this outcome. 


18 The final study outcomes were categorized into standard AAPOR disposition codes as follows: (1) Interview conducted includes cases 
where the visit resulted in a completed interview. (2) Eligible, non-interview includes short interview completed, soft refusal, and hard 
refusal. (3) Unknown eligibility includes physical or mental disability, language barrier, unable to reach household or unsafe, no one 
home, and ineligible person and (4) /neligible includes ineligible household and vacant or non-existent units. 
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Table E.5. Number and percentage distribution of final study outcome for 
addresses sampled for interview component: 2019 


Percentage of 


Final study outcome Number of addresses addresses 

Total 400 100.0 
Interview completed 85 21.3 
Short interview completed 6 1.5 
Soft refusal 47 11.8 
Hard refusal 75 18.8 
Physical or mental disability 2 0.5 
Language barrier 10 2.5 
Unable to reach household or unsafe 40 10.0 
No answer 97 24.3 
Ineligible person 22 5.5 
Ineligible household 2 0.5 
Vacant or non-existent unit 14 3.5 


NOTE: The final study outcome is the final disposition status assigned to a case based the combined outcomes of all recruitment efforts, including in- 
person visits. Details may not sum to totals because of rounding. Sample sizes have not been rounded. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019. 
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interview component, by selected characteristics: 2019 


Selected characteristics 


Total 
Age of head of household1 
18-34 
35-44 
45-54 
55-64 
65 and older 
Missing 
Gender of head of household! 
Male 
Female 
Missing 
Education of head of household1 
Less than high school credential 
High school credential 
Some college 
Bachelor's degree 
Graduate degree 
Missing 
Race of head of household! 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other race, non-Hispanic 
Missing 


Number 
of addresses 


400 


38 
63 
59 
61 
54 
125 


129 
125 
146 


120 
71 
71 
50 
22 
66 


145 
55 
105 
25 
70 


Interview 
conducted 


21.3 


21.1 
27.0 
18.6 
26.2 
18.5 
18.4 


20.9 
20.8 
21.9 


23.3 
25.4 
25.4 
18.0 
22.7 
10.6 


22.8 
29.1 
21.0 
20.0 
12.9 
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Final study outcome 


Eligible Unknown 
non-interview eligibility 
32.0 42.8 
34.2 42.1 
28.6 42.9 
475 32.2 
31.2 37.7 
35.2 42.6 
24.8 50.4 
31.0 43.4 
38.4 37.6 
27.4 46.6 
26.7 44.2 
38.0 32.4 
32.4 40.9 
42.0 38.0 
36.4 40.9 
25.8 57.6 
36.6 37.9 
21.8 43.6 
35.2 39.1 
28.0 48.0 
27.1 55.7 


Ineligible 
4.0 


2.6 
1.6 
1.7 
4.9 
3.7 
6.4 


4.7 
3.2 
4.1 


5.8 
4.2 
1.4 
2.0 
0.0 
6.1 


2.8 
5.5 
48 
4.0 
4.3 


See notes at end of table. 
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interview component, by selected characteristics: 2019—Continued 


Selected characteristics 


Household income! 
Less than $50,000 
$50,000-$74,999 
$75,000-$99,999 
$100,000 or higher 
Missing 

Household flagged as having children1 
Yes 
No 

Number of adults in household! 
1 adult 
2 adults 
3 adults or more 
Missing 

Phone number available! 
Yes 
No 

Route type? 
Street 
High rise 

Dwelling type1 
Single-unit 
Multi-unit 

Home tenure! 
Own 
Rent 
Missing 


Number 
of addresses 


192 
46 
42 
85 
35 


120 
280 


233 
99 
33 
35 


273 
127 


274 
126 


259 
141 


200 
149 
51 


Interview 
conducted 


21.9 
26.1 
26.2 
17.7 
14.3 


21.7 
21.1 


22.8 
17.2 
30.3 
14.3 


24.2 
15.0 


20.4 
23.0 


20.9 
22.0 


18.5 
28.2 
11.8 


Final study outcome 


Eligible 
non-interview 


28.7 
37.0 
35.7 
40.0 
20.0 


32.5 
31.8 


29.2 
39.4 
42.4 
20.0 


35.9 
23.6 


35.4 
24.6 


36.3 
24.1 


38.5 
28.2 
17.7 


Unknown 
eligibility 


44.8 
32.6 
38.1 
38.8 
60.0 


43.3 
42.5 


43.4 
40.4 
27.3 
60.0 


36.6 
55.9 


39.4 
50.0 


38.2 
51.1 


41.5 
38.9 
58.8 


Ineligible 


4.7 
4.4 
0.0 
3.5 
5.7 


2.5 
4.6 


4.7 
3.0 
0.0 
5.7 


3.3 
5.5 


4.7 
2.4 


4.6 
2.8 


1.5 
4.7 
11.8 


See notes at end of table. 
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Table E.6. Number of addresses and percentage distribution of final study outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Selected characteristics 
Urbanicity! 
Urban 
Suburban 
Town 
Rural 
Region! 
Northeast 
South 
Midwest 
West 
Race/ethnicity stratum2 
25% or more Black 
40% or more Hispanic 
Other 
Tract poverty rate2 
Less than 20% 
20% or more 
Percent of households in Census block that include a child? 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 
Percent of persons in Census block that speak a language 
other than English2 
First quartile 
Second quartile 
Third quartile 
Fourth quartile 


Number 
of addresses 


121 


107 
174 


Interview 
conducted 


24.2 
20.4 
0.0 
6.3 


20.0 
23.0 
28.0 
14.0 


35.6 
19.7 
17.0 


18.2 
26.3 


19.4 
26.4 
15.2 
24.0 


29.3 
21.8 
19.6 
20.1 


Final study outcome 


Eligible 
non-interview 


26.7 
35.7 

0.0 
43.8 


39.0 
33.0 
20.0 
36.0 


16.4 
35.4 
35.5 


37.1 
23.7 


33.3 
39.1 
26.3 
30.6 


22.0 
25.6 
40.2 
32.2 


Unknown 
eligibility 


44.9 
39.8 
100.0 
50.0 


36.0 
41.0 
49.0 
45.0 


42.5 
42.5 
43.0 


40.7 
46.1 


46.2 
33.3 
48.5 
42.2 


41.5 
52.6 
35.5 
43.1 


Ineligible 


4.2 
4.2 
0.0 
0.0 


5.0 
3.0 
3.0 
5.0 


5.5 
24 
4.5 


4.0 
4.0 


11 
1.2 
10.1 
3.3 


7.3 
0.0 
4.7 
4.6 


See notes at end of table. 
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Table E.6. Number of addresses and percentage distribution of final study outcome for addresses sampled for interview 
component, by selected characteristics: 2019—Continued 


Final study outcome 


Number Interview Eligible Unknown 
Selected characteristics of addresses conducted non-interview eligibility Ineligible 
Percent of persons in Census block without a high school 
diploma or the equivalent2 
First quartile 82 14.6 34.2 47.6 3.7 
Second quartile 78 20.5 30.8 44.9 3.9 
Third quartile 81 22.2 38.3 37.0 2.5 
Fourth quartile 159 24.5 28.3 42.1 5.0 
Low Response Score? 
First quartile 86 15.1 41.9 39.5 3.5 
Second quartile 70 15.7 34.3 47.1 2.9 
Third quartile 78 30.8 29.5 35.9 3.9 
Fourth quartile 165 22.4 27.3 45.5 4.9 
Residential high-speed internet per 1000 households* 
600 or less 116 25.9 23.3 46.6 4.3 
601-800 150 21.3 35.3 40.0 3.3 
801 or more 134 17.2 35.8 42.5 4.5 


1These characteristics are based on variables available on the NHES:2019 sampling frame. 

2These characteristics are based on American Community Survey (2013-2017) five-year estimates. Cases in the first quartile were those with the lowest prevalence of the characteristic in question and those that are in the 
fourth quartile are those with the highest prevalence. 

3The Low Response Score is a derived variable that identifies block groups with characteristics associated with low mail return rates to the 2010 Decennial Census. A higher score corresponds to a lower expected mail return 
rate. This variable was not available for a very small number of cases, and these cases have been excluded from this analysis. 

4This characteristic is based on tract-level estimates of internet penetration provided by the Federal Communications Commission (FCC). 

NOTE: The final study outcome for each case was categorized into American Association for Public Opinion Research (AAPOR) standard disposition outcomes. The categorization was conducted as follows: (1) Interview 
conducted includes cases where the visit resulted in a completed interview. (2) Eligible, non-interview includes short interview completed, soft refusal, and hard refusal. (3) Unknown eligibility includes physical or mental 
disability, language barrier, unable to reach household or unsafe, no one home, and ineligible person. (4) /neligible includes ineligible household and vacant or non-existent units. Details may not sum to totals because of 
rounding. Sample sizes have not been rounded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 2019; U.S. Department of Commerce, American Community Survey (ACS), 2013-2017 
and Decennial Census, 2010; and Federal Communications Commission (FCC), 2017. 


E-21 


Appendix F. NHES:2019 Materials 


Appendix F. NHES:2019 Materials 
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Advance letter en oF OH, UNITED STATES DEPARTMENT OF COMMERCE 
Economics and Statistics Administration 

U.S. Census Bureau 

Washington, DC 20233-0001 


OFFICE OF THE DIRECTOR 


© 


NHES-10L(MW) 
(06-2018) 


e ss 
Steg . 208" 


| am pleased to inform you that your household has been selected to participate in an 
important national survey on the education of people in the United States, the 2019 
National Household Education Survey. The U.S. Census Bureau, on behalf of the U.S. 
Department of Education, conducts this survey of households every few years to 
gather information about learning activities that happen outside of schools. 


What to expect next: 


e Ina few days you will receive an invitation in the mail from us to complete the 
survey. 
The survey can be completed at home by any adult in the household. 
Depending on your answers, it takes about 3 to 30 minutes to complete. 
We include $5 as a token of our appreciation for your participation in this study. 
Please look for it in your next mailing from us! 


If you would like more information on the study, please visit nces.ed.gov/nhes or 
contact us toll-free at 1-888-840-8353. 


Thank you in advance for your help with this important study. 


Sincerely, 


/ 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 


Cens States” 


que Bureau census.gov 
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Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is 
authorized to conduct the National Household Education Survey (NHES) by the Education Sciences 
Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this 
voluntary survey on behalf of NCES. There are no penalties should you choose not to participate in this 
study. All of the information you provide may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 
and 6 U.S.C. §151). 


NHES-10L(MW) (06-2018) = 


Initial screener package 
\%, | UNITED STATES DEPARTMENT OF COMMERCE 
W Economics and Statistics Administration 
U.S. Census Bureau 
Washington, DC 20233-0001 
OFFICE OF THE DIRECTOR 


NHES-11L(W) 
(06-2018) 


Ur 
& 
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Last week, the U.S. Census Bureau sent a letter to your household about the 2019 
National Household Education Survey. Please complete this survey online as soon as 
possible using the instructions below. 


Follow these three easy steps: 


1. Go to: https://respond.census.gov/nhes 


2. Enter your User ID: 
3. Click "Begin survey" to participate in the survey. 


We know that you are busy, and we have enclosed a $5 token of our appreciation for 
your time. 


We understand that not all households have Internet access. If you want the survey 
in another format, have questions, or need assistance, please contact the U.S. 
Census Bureau toll-free at 1-888-840-8353. You can find more information online at 
https://nces.ed.gov/nhes/. 


Thank you for your participation in this valuable national survey. We look forward to 
your response. 


Sincerely, 


j 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 


Enclosures 


United States” 
C F-4 


epee Bureau census.gov 


Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is 
authorized to conduct the National Household Education Survey (NHES) by the Education Sciences 
Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this 
voluntary survey on behalf of NCES. There are no penalties should you choose not to participate in this 
study. All of the information you provide may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 
and 6 U.S.C. §151). 


NHES-11L(W) (06-2018) — 


Pressure-sealed envelope 
geNT OF Coy 


“a UNITED STATES DEPARTMENT OF COMMERCE 


yo 


0939-1 2P(W) ws Economics and Statistics Administration 
(06-2018) 5 U.S. Census Bureau 
ks €& Washington, DC 20233-0001 
Stares oF " OFFICE OF THE DIRECTOR 


Recently we mailed you a letter asking you to participate in the 2019 National Household Education 
Survey. If you or someone in your household has already completed the survey, we thank you very 
much for your help! If you have not completed your survey, please do so right away. 


Follow these steps to finish the survey online: 


1. Go to: https://respond.census.gov/nhes 
2. Enter your User ID: 


3. Click "Begin survey" to participate in the survey. 


If you have questions, feel free to contact us toll-free at 1-888-840-8353. 


Sincerely, 


/ 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 


Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is authorized to 
conduct the National Household Education Survey (NHES) by the Education Sciences Reform Act of 2002 (ESRA 
2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this voluntary survey on behalf of NCES. There 
are no penalties should you choose not to participate in this study. All of the information you provide may be used 
only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except 
as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). 
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U.S. DEPARTMENT OF COMMERCE 
Economics and Statistics Administration 
U.S. Census Bureau 

1201 E 10th St 

Jeffersonville IN 47132-0001 


OFFICIAL BUSINESS 
Penalty for Private Use $300 


0939-1 2P(W) (06-2018) 


AN EQUAL OPPORTUNITY EMPLOYER 
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PRESORTED 
FIRST-CLASS MAIL 
POSTAGE & FEES PAID 
U.S. Census Bureau 
Permit No. G-58 
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Second screener package: FedEx version 


Ow UNITED STATES DEPARTMENT OF COMMERCE 


£ Economics and Statistics Administration 
NHES-12L(SM) = U.S. Census Bureau 
(08-2018) > Washington, DC 20233-0001 


‘Oo 


<p RS 
Stares of © OFFICE OF THE DIRECTOR 


We have mailed this survey using rush delivery because of the importance of your response. We 
have not yet received your completed survey for the 2019 National Household Education Survey. 
Did you know? 


e From 2005 to 2016, the cost of child care from a day care center, Head Start program, or 
preschool rose from an average of $5.29 to $7.60 per hour. 


e From 2012 to 2016, the percentage of parents who reported that the amount of homework 
their child was assigned was “about right” declined from 77% to 75%. 


These are just some of the statistics that we learn from the National Household Education Survey. 
Topics include: 
Early childhood care and education | Parent and family involvement in education | Homeschooling 


Now, we need your help to update this important information. Follow these steps to complete the 
survey: 


1. Go to: https://respond.census.gov/nhes 
2. Enter your User ID: 


3. Click "Begin survey" to participate in the survey. 


If you have questions or need assistance, please contact us toll-free at 1-888-840-8353. You 
can find more information online at https://nces.ed.gov/nhes/. 


Thank you for taking part in this important study. 


Sincerely, 


/ 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 
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Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is 
authorized to conduct the National Household Education Survey (NHES) by the Education Sciences 
Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this 
voluntary survey on behalf of NCES. There are no penalties should you choose not to participate in this 
study. All of the information you provide may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 
and 6 U.S.C. §151). 


NHES-12L(SM) (08-2018) - 


UNITED STATES DEPARTMENT OF COMMERCE 
Economics and Statistics Administration 

U.S. Census Bureau 

Washington, DC 20233-0001 

OFFICE OF THE DIRECTOR 


NHES-1 2L(W) 
(08-2018) 


Did you know? 


e From 2005 to 2016, the cost of child care from a day care center, Head Start program, or 
preschool rose from an average of $5.29 to $7.60 per hour. 


e From 2012 to 2016, the percentage of parents who reported that the amount of homework their 
child was assigned was “about right” declined from 77% to 75%. 


These are just some of the statistics that we learn from the National Household Education Survey. 
Topics include: 
Early childhood care and education | Parent and family involvement in education | Homeschooling 


Now, we need your help to update this important information. Follow these steps to complete the 
survey: 


1. Go to: https://respond.census.gov/nhes 
2. Enter your User ID: 


3. Click "Begin survey" to participate in the survey. 


If you have questions or need assistance, please contact us toll-free at 1-888-840-8353. 
You can find more information online at https://nces.ed.gov/nhes/. 


Thank you for taking part in this important study. 


Sincerely, 


i 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 
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Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is 
authorized to conduct the National Household Education Survey (NHES) by the Education Sciences 
Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this 
voluntary survey on behalf of NCES. There are no penalties should you choose not to participate in this 
study. All of the information you provide may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 
and 6 U.S.C. §151). 
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We have not yet received your completed survey for the 2019 National Household 
Education Survey. 


Please fill out the enclosed survey and return it in the postage-paid envelope as soon 
as possible. 


Did you know? 

All households that were selected for the survey that do not respond harm the 
accuracy of the study’s findings. Whether your household is large or small, has children 
or not, your response is needed to provide the best possible data about education in 
America. 


Call us toll-free at 1-888-840-8353 with any questions or to complete the survey with 
an interviewer. If there are no children in your household, this call will take 5 minutes 
or less. 

Thank you for taking part in this important study. 


Sincerely, 


/ 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 


Enclosures 


Cens States” 
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Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is 
authorized to conduct the National Household Education Survey (NHES) by the Education Sciences 
Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this 
voluntary survey on behalf of NCES. There are no penalties should you choose not to participate in this 
study. All of the information you provide may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 
and 6 U.S.C. §151). 
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Fourth screener package: First-Class Mail version penis 
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The last mailing we sent you was sent by rush delivery because of the importance of 
YOUR household’s response to the 2019 National Household Education Survey. 


It will take 3 minutes or less to complete the survey. 

If you have any questions or would like to complete the survey with an interviewer, 
please call us toll-free at 1-888-840-8353. Please respond within the next TWO 
WEEKS. 

You can find more information about this study online at nces.ed.gov/nhes. 


Thank you for taking part in this important study. 


Sincerely, 


/ 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 
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Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is 
authorized to conduct the National Household Education Survey (NHES) by the Education Sciences 
Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this 
voluntary survey on behalf of NCES. There are no penalties should you choose not to participate in this 
study. All of the information you provide may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 
and 6 U.S.C. §151). 
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We have mailed this survey using rush delivery because of the importance of your 
response to the 2019 National Household Education Survey. 


It may take 3 minutes or less to complete the survey. 

If you have any questions or would like to complete the survey with an interviewer 
please give us a Call toll-free at 1-888-840-8353. Please respond within the next 
TWO WEEKS. 

You can find more information about this study online at nces.ed.gov/nhes. 


Thank you for taking part in this important study. 


Sincerely, 


/ 


Ron S. Jarmin 
Performing the Non-Exclusive Functions 
and Duties of the Director, U.S. Census Bureau 
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Authorization and Confidentiality of Information 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is 
authorized to conduct the National Household Education Survey (NHES) by the Education Sciences 
Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The U.S. Census Bureau is administering this 
voluntary survey on behalf of NCES. There are no penalties should you choose not to participate in this 
study. All of the information you provide may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 
and 6 U.S.C. §151). 
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US DEPARTMENT OF COMMERCE AN EQUAL OPPORTUNITY EMPLOYER SRESDRTED 
Economics and Statistics Administration FIRST-CLASS MAIL 
Ve conse Bureau POSTAGE & FEES PAID 
th Street U.S. Census Bureau 
Jeffersonville IN 47132-0001 Permit No. G-58 


OFFICIAL BUSINESS 
Penalty for Private Use $300 
BC-1328(0977) (3-2018) 
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Envelope for third screener package and fourth screener package (First-Class Mail version) 


US DEPARTMENT OF COMMERCE AN EQUAL OPPORTUNITY EMPLOYER PRESORTED 
Economics and Statistics Administration FIRST-CLASS MAIL 
US Census Bureau 

1201 E 107TH Street POSTAGE & FEES PAID 
Jeffersonville IN 47132-0001 U.S. CENSUS BUREAU 
OFFICIAL BUSINESS PERMIT NO. G-58 
Penalty for Private Use $300 

BC-1728(0939S) (8-2018) 


Please respond within two weeks. 
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Envelope for FedEx version of second screener package and fourth screener package 
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> PP Pull to open Customer: These service boxes are provided for your convenience. Service must be marked on airb!I. Pull to open qq < 


Extremely Urgent: ia FedEx Priority Overnight® ia FedEx Standard Overnight® CJ FedEx First Overnight™ i FedEx International Services 7 Saturday Delivery 


Recip ent please hand deliver to Next business morming service Next business day service Earliest next business maming Scheduled delivery times vary by lacation. Service not available to 
addressee. (not avaitable to all locations). (not available to all locations). delivery to select locations. all locations. 
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OMB No. 1850-0768: Approval Expires 09/30/2021 


get OF Co 
UNITED STATES DEPARTMENT OF COMMERCE . W “ 
Economics and Statistics Administration 
U.S. Census Bureau 


National Household Education Survey 


The National Center for Education Statistics (NCES), within the U.S. Department of Education, is authorized to conduct the National Household Education 
Survey (NHES) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. 89543). The U.S. Census Bureau is administering this voluntary 
survey on behalf of NCES. There are no penalties should you choose not to participate in this study. All of the information you provide may be used 
only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. 89573 
and 6 U.S.C. 8151). According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays 
a valid OMB control number. The valid OMB control number for this voluntary survey is 1850-0768. The time required to complete this survey is estimated 
to average 3 minutes per response, including the time to review instructions, gather the data needed, and complete and review the survey. If you have 
any comments concerning the accuracy of the time estimate, suggestions for improving this survey, or any comments or concerns regarding the status 
of your individual submission of this survey, please e-mail: nhes@census.gov or write directly to: Sarah Grady, National Center for Education Statistics 
(NCES), PCP, 550 12th St., SW, 4th floor, Washington, DC 20202. 
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Conducted for: 
U.S. Department of Education 


National Center for Education Statistics 


National Household Education Survey 


> Start with the youngest child or 


Child / Youth 1 Child / Youth 2 Child / Youth 3 Child / Youth 4 Child / Youth 5 


Start Here 


The Department of Education is studying 
households with children or youth age 20 or 
younger. Each household is different, and 
we need your response so we can send you 
a survey that is right for your household. 


> Return this form even if there are no 
children or youth in this household after 
marking the correct box in item 1 below. 


This survey should be filled out by an 
adult household member living at this 
address. 


Please use a blue or black pen if available. 


. Are there any children or youth age 
20 or younger living in this household? 


Include small children, foster children, 
babies, and those living in college 
housing (if they have no other permanent 
home). 


L] Yes 
Please stop here and 

[| No ==» RETURN this survey to us 
in the enclosed envelope. 
It is important that we 
receive a response from 
every household selected 
for this study. Thank you 
for your time. 


. How many children or youth age 20 
or younger live in this household? 


oo number age 20 or younger 


Continue answering questions 3 
through 7 for each child or youth 
living in this household. 


NHES-SCRN (09/18/2018) 


youth who is age 20 or younger. 


. What is his or her first name, 
initials, or nickname? 
First names will be used only 
to ask you questions about the 
education of a specific child. 


. What is this child/youth’s 
month and year of birth? 


. What is this child/youth’s sex? 


. Is this child/youth 
currently in 


Mark [|X] ONE only. 


7. What is this child/youth’s 
current grade or equivalent? 


Vv 


Vv 


Vv 


Vv 


Vv 


First name/initials/nickname 


month 


O 
O 


year of birth 


Male 


Female 


Homeschool instead 
of attending a public 
or private school for 
some or all classes, 


Public or private 
school, or preschool, 


College, university or 
vocational school, or 


Not in school? 


L GO TO child/youth 2 


O 
O 


O 
O 


Preschool 


Kindergarten 


write grade 
1 through 12 


College, university or 
vocational school 


None of these 


First name/initials/nickname 


month 


al 
O 


year of birth 


Male 


Female 


Homeschool instead 
of attending a public 
or private school for 
some or all classes, 


Public or private 
school, or preschool, 


College, university or 
vocational school, or 


Not in school? 


L GO TO child/youth 3 


O 
O 


Preschool 


Kindergarten 


write grade 
1 through 12 


O 
O 


College, university or 
vocational school 


None of these 


First name/initials/nickname 


month 


a 
O 


year of birth 


Male 


Female 


Homeschool instead 
of attending a public 
or private school for 
some or all classes, 


Public or private 
school, or preschool, 


College, university or 
vocational school, or 


Not in school? 


L GO TO child/youth 4 


O 
O 


Preschool 


Kindergarten 


write grade 
1 through 12 


O 
O 


College, university or 
vocational school 


None of these 


First name/initials/nickname 


month 


O 
oO 


year of birth 


Male 


Female 


Homeschool instead 
of attending a public 
or private school for 
some or all classes, 


Public or private 
school, or preschool, 


College, university or 
vocational school, or 


Not in school? 


L GO TO child/youth 5 


O 
O 


Preschool 


Kindergarten 


write grade 
1 through 12 


O 
O 


College, university or 
vocational school 


None of these 


> Please verify you have listed the 5 youngest children or youth living in this household in columns 1 through 5 above. 


First name/initials/nickname 


month 


al 
EB 


year of birth 


Male 


Female 


Homeschool instead 
of attending a public 
or private school for 
some or all classes, 


Public or private 
school, or preschool, 


College, university or 
vocational school, or 


Not in school? 


Ly Return Survey. 


O 
O 


Preschool 


Kindergarten 


write grade 
1 through 12 


O 
O 


College, university or 
vocational school 


None of these 


> Thank you. Please return this form in the postage-paid envelope provided or mail it to: 
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U.S. Census Bureau 


ATTN: DCB 60-A (0939) 


1201 E. 10th Street 


Jeffersonville, IN 47132-0001 


Toll-free number for questions: 1-888-840-8353 


24019028 
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Appendix G. Interview Materials 


G.1 Materials Used for Recruitment 


G.1.1 Invitation Letter 


U.S. DEPARTMENT OF EDUCATION 
INSTITUTE OF EDUCATION SCIENCES 


NATIONAL CENTER FOR EDUCATION STATISTICS 


Dear <City> Household: 


The National Center for Education Statistics (NCES) and American Institutes for Research (AIR) are 
doing an important study in your area. 


e = This study will help us understand how to make NCES research more useful and helpful. We would like 
to get your feedback on a variety of topics, such as how you like to spend your time, how you go about 
day-to-day tasks like sorting your mail, and how you feel about participating in research. 


e Because your participation is so important, you will get $120 cash as a thank you for sharing your 
opinions and experiences with us. This voluntary, 90-minute conversation takes place in your home 
at a convenient time for you. 


We know that you are busy, and we have enclosed $5 as a thank you for your time. 


To see if you are eligible to participate, please do one of the following: 


e = Call us toll-free at 1-888-261-0922. 
e E-mail us at helpNCES@air.org. 


Thank you in advance for participating. We look forward to hearing from you. 


Sincerely, 
sat Ldn 


James Lynn Woodworth 
Commissioner, National Center for Education Statistics 


Authorization and Confidentiality of Information 


American Institutes for Research is administering this voluntary study on behalf of the National Center for Education Statistics (NCES). 
There are no penalties should you choose not to participate in this study. All of the information you provide may be used only for statistical 
purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 
U.S.C. §151). The National Center for Education Statistics (NCES), within the U.S. Department of Education, is authorized to conduct the 
National Household Education Survey (NHES) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). 
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G.1.2 First Reminder Postcard 


Would you like $120 for sharing your 
Opinions? 


The National Center 


10) a =o [0(er=) (0) a) 


Statistics (NCES) 
and American 
Institutes for 
Research (AIR) are 
folo)iatcar= lamina) exe)atelale 
study in your area. 
This study will help 
UiSMUlaleletecirel aremarey i 
to make NCES 
research more 


useful and helpful. 


Because your participation is so critical, you will get $120 cash 
as a thank you for sharing your opinions and experiences with us. 


This voluntary, 90-minute conversation takes place in your home ata 
convenient time for you. 


We would like to get your feedback on a variety of topics, such as how 
you like to spend your time, how you go about day-to-day tasks like 
sorting your mail, and how you feel about participating in research. 


A researcher from AIR will stop by in the next few weeks to invite you to 
participate. 


To find out sooner if you are eligible to participate: call us toll-free at 
1-888-261-0922 or e-mail us at helpNCES@air.org. 


Thank you in advance for participating. We look forward to 
hearing from you! 
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G.1.3. Second Reminder Postcard 


Chat with us to help make research 
more useful! 


The National Center 


for Education 


Statistics (NCES) 
and American 


Institutes for 


An AIR researcher will stop by in the next few weeks to invite 
you to participate. 


To find out sooner if you are eligible to participate: call us toll-free at 


1-888-261-0922 or e-mail us at helpNCES@air.org. 
Research (AIR) are 
We would like to get your feedback on a variety of topics, such as how 


you like to spend your time, how you go about day-to-day tasks like 
sorting your mail, and how you feel about participating in research. 


(ole) igrearelam lanl ele)arclale 


study in your area. 


Because your participation is so important, you will get $120 cash as a 
thank you for sharing your opinions and experiences with us. 


This study will help 


us understand how : ; : ; 
This voluntary, 90-minute conversation takes place in your home ata 


to make NCES convenient time for you. 


eee len ne Thank you in advance for participating. We look forward to 


useful and helpful. hearing from you! 
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G.1.4 Sorry We Missed You Card 


Sorry We Missed You! 


The National Center 


for Education e This study will help us understand how to make 
Statistics (NCES) NCES research more useful and helpful. We would like 
to get your feedback on a variety of topics, such as how you 
and American . : . 
like to spend your time, how you go about day-to-day tasks like 


Institutes for sorting your mail, and how you feel about participating in research. 
Research (AIR) are e If you meet with us, you will receive $120. 

doing an important e This voluntary conversation lasts about 90 minutes and takes 
study in your area. place in your home at a convenient time for you. 

tek AM Contact us at 1-888-261-0922 or helpNCES@air.org to see if 
invite you to you are eligible! 


participate. 
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G.1.5 Appointment Reminder Card 


Your conversation with 
American Institutes for Research 


Conversation Day and Date: 
Conversation Time: 
Conversation With: 


Questions? 
Call us at 1-888-261-0922 or e-mail us at helpNCES@air.org. 


m AIR 


(AMERICAN INSTITUTES FOR RESEARCH’ 
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G.1.6 Commonly Asked Questions Handout 


ml AIR 


AMERICAN INSTITUTES FOR RESEARCH 


Commonly Asked Questions 


What is this study about? What will you do with my responses? 
The focus of the study is to understand how to improve research done by the National Center for Education Statistics, 


which is part of the U.S. Department of Education. We would like to hear about your opinions and experiences with a 
variety of topics. For example, we would like to know what kinds of things in your everyday life are important to you. 
We would also like to know about how you handle certain tasks, like collecting and reviewing your home’s postal mail. 
Finally, we would like to get your feedback on some of our research materials. Ultimately, this information will help us 
improve the National Household Education Surveys and ensure that they are relevant to all household in the United 
States. 


Who is conducting this study? 
The U.S. Department of Education, specifically the National Center for Education Statistics (NCES), is conducting this 


study. You can learn more about NCES by going to www.nces.ed.gov. 


The American Institutes for Research (AIR), a not-for-profit behavioral and social science research firm, is carrying 
out this study on behalf of NCES. You can learn more about AIR by going to www.air.org. 


How much time will this take? 
The conversation should take about 90 minutes to complete. 


Where will this take place? 
The conversation will take place at your home ata time that is convenient for you. We would like to speak with you in 


your home because we want you to share your experiences and opinions in a setting that is comfortable and familiar 
to you. 


Who will know if I participate in the study? 
Only the study team at AIR will know who participated in the study. The information collected during our 


conversation will be stored on AIR’s secure computer servers. We will not link what you say to anything identifiable 
about you, such as your name or address. 


What do I get if I meet with you? 
You will receive $120 cash as a thank you for your participation. 


Who can I contact if I have questions about participating in the study? 
You can call us toll-free at 1-888-261-0922 or e-mail us at helpNCES@air.org. 
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G.1.7 Telephone Recruitment Script 


Introduction/recruitment script 
IF SOMEONE ANSWERS THE PHONE: 


Hello, this is [NAME] from American Institutes for Research, calling on behalf of the National 
Center for Education Statistics. Am I speaking with a resident of [ADDRESS]? 


e Ifyes, continue with the next paragraph. 
e /fno, say:1am sorry. I have the wrong number. Thank you for your time. Have a nice 
day. End phone call. 


I am calling to follow up on a letter that was recently mailed to your house. The National 
Center for Education Statistics and American Institutes for Research are conducting an 
important study in your area that will help us understand how to make our research more 
useful and helpful. We would like to meet with you to get your feedback on a variety of topics, 
such as how you like to spend your time and how you go about daily tasks like sorting mail. 
If you complete a 90-minute conversation with us in your home, we will give you $120 cash 
to show our thanks. 


Are you interested in this opportunity? 
e Ifyes, continue with the next paragraph. 
e Ifno, try to address their concerns. If still not interested, say: No problem. Thank you 
for your time. Have a good day. End phone call. 


Screening script 


Great. First, I need to ask a few questions to make sure you are eligible to participate. 


1. Did you live or stay here most of the time between January and April 2019? 

e Ifyes, continue with next screener item. 

e /fno, say: Unfortunately, we can only meet with people who lived or stayed at 
this address most of the time between January and April 2019. Is there 
anyone else living here who lived or stayed here most of that time? 

= If yes, ask ifyou can speak with them about participating. 

e When that person is on the phone, go back to the study 
introduction paragraph and then restart the screener. 

e Ifthe person is not available right now, ask when would be a 
good time for you to call back, thank them for their time, and 
then end the call. Try to get the other person’s first name to 
facilitate the return call. 

# Ifno one else lived here during that time period, thank them for their 
time, end phone call and do not call again. 


2. Ok, great. Can you please confirm your age? 
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e If 18 or older, continue with next screener item. 
e If 17 or younger, say: Unfortunately, only adults can participate. Is there an 
adult at home with whom I can speak? 

= When the adult is on the phone, go back to the study introduction 
paragraph and then restart the screener. 

# [fan adult is not available right now, ask when would be a good time for 
you to call back to speak with one, thank them for their time, and then 
end the call. Try to get the other person’s first name to facilitate the 
return call. 


3. Great, thanks. Are you involved in handling your household’s postal mail—for 
example, collecting, reviewing, or responding to the mail? 
e Ifyes, continue with next screener item. 
e /fno, say: Unfortunately, we can only meet with one of the people who 
handles your household’s mail. Can I please speak to one of those people? 
= When that person is on the phone, go back to the study introduction 
paragraph and then restart the screener. 
= Ifthe person is not available right now, ask when would be a good time 
for you to call back, thank them for their time, and then end the call. Try 
to get the other person’s first name to facilitate the return call. 


Great, thank you! You are eligible to participate in the study. 

Scheduling script 

What days [this/next] week would be convenient for you to meet with us? 

Looking at when interview teams are available in that area, work with participant to schedule 


the interview. Our preference is to do interviews sooner than later. 


Once interview date/time are decided, record them and say: We have you scheduled on 
[DATE] at [TIME]. 


[Note: the possible dates for doing interviews are: May 3-8, May 15-21, May 30-June 5. We 
prefer to schedule something as soon as possible, in case the participant ends up having a 
conflict and needs to reschedule. | 


We would like to send you a confirmation message with the date and time of your meeting. 
Do you prefer that we text you or that we e-mail you? 
e If willing to receive confirmation message: And what is the best [phone number/e- 
mail address] for reaching you? 
e Ifnot willing to receive confirmation message, reassure them that this information will 
only be used for communicating with them about the interview. If still not willing, say: 
Ok, no problem. 
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We would also like to reach out to you the day before the meeting to confirm that this time 
still works for you. Do you prefer that we call you or that we e-mail you? 


e I[fwilling to receive reminder message and request a different mode than for the 
confirmation message: And what is the best [phone number/e-mail address] for 
reaching you? 

e Ifnot willing to receive reminder message, reassure them that this information will 
only be used for communicating with them about the interview. If still not willing, say: 
Ok, no problem. 


Thank you for participating! If any conflicts come up between now and [DATE OF 
INTERVIEW] please call our toll-free number at 1-888-261-0922 or email us at 
HelpNCES@air.org. 


We look forward to seeing you soon. Bye. End call. 


G.1.8 In-Person Recruitment Script 
Introduction 


If the person who answers the door is clearly a child, ask about speaking to an adult in the 
house. Otherwise, say: 


Hi, my name is [YOUR NAME] from American Institutes for Research. We recently sent a 
letter to your house about an important study we are doing with the National Center for 
Education Statistics. We would like to have a conversation with you to get your opinions on 
a variety of topics, such as how you like to spend your time, how you go about day-to-day 
activities like sorting your mail, and how you feel about participating in research. The 
conversation lasts 90 minutes, and we will give you $120 cash as a thank you for your time. 
Would you like to participate? 


e If yes, continue with the next section. 
e Ifno, try to address their concerns. If still not interested, say: No problem, I understand. Is 
anyone else from this household available to speak with us? 
« Ifyes and available, repeat the introduction once that person comes to the door. 
« Ifyes but not available right now, say: No problem. We can come back at a more 
convenient time for them. What would be a good time for us to come back? 
[record date/time, and, if possible, the other household member’s first name to 
facilitate the return visit] Here is a card with more information about the study 
that you can give to them. It also lists our contact information in case they would 
like to contact us sooner about participating. [hand Sorry We Missed You card to 
household member] Thank you for your time! Have a nice day. 
= Ifno one else lives here or not interested in you speaking with anyone else, say: | 
understand. Would you be willing to answer a few quick questions instead? It 
will take less than five minutes. 
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= Ifno: No problem. Thank you for your time. Have a nice day. (end 
interview) 
= Ifyes, go to Short Interview Questions: Great, thank you. 


Screener 


Once the interested household member is at the door: 
Great! First, I need to ask a few quick questions to make sure you are eligible to participate. 
1. Can you please confirm that we are at [STREET ADDRESS]? 
e Ifyes, continue with next screener item. 
e /fno, say: Unfortunately, I can only meet with someone who lives at [STREET 
ADDRESS]. I am sorry; I must have the wrong house. Thank you for your 
time. Have a nice day. End conversation. 


2. Did you live or stay here most of the time between January and April 2019? 

e Ifyes, continue with next screener item. 

e /fno, say: Unfortunately, we can only meet with people who lived or stayed at 
this address most of the time between January and April 2019. Is there 
anyone else living here who lived or stayed here most of that time? 

= Ifyes and available, repeat the introduction once that person comes to 
the door. 

= Ifyes but not available right now, say: No problem. We can come back 
at a more convenient time for them. What would be a good time for us 
to come back? [record date/time, and, if possible, the other household 
member’s first name to facilitate return visit] Here is a card with more 
information about the study that you can give to them. It also lists our 
contact information in case they would like to contact us sooner about 
participating. [hand Sorry We Missed You card to household member] 
Thank you for your time! Have a nice day. 

# Ifno one else lives here or not interested in you speaking with anyone 
else, say: No problem. Thank you for your time. Have a good day. 


3. Ok, great. Can you please confirm your age? /do not ask this if respondent is clearly 
much older than 18, but if there is any uncertainty make sure you confirm their age] 
= If 18 or older, continue with next screener item. 
« If 17 or younger, say: Unfortunately, only adults can participate. Is there an 
adult at home with whom I can speak? 
# Ifyes and available, repeat the introduction once that person comes to 
the door. 
« Ifyes but not available right now, say: No problem. We can come back 
at a more convenient time for them. What would be a good time for us 
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to come back? [record date/time, and, if possible, the other household 
member's first name] Here is a card with more information about the 
study that you can give to them. It also lists our contact information in 
case they would like to contact us sooner about participating. /hand 
Sorry We Missed You card to household member] Thank you for your 
time! Have a nice day. 

= Ifno one else lives here or not interested in you speaking with anyone 
else, say: No problem. Thank you for your time. Have a good day. 


4. One last question: Are you involved in handling your household’s postal mail—for 
example, collecting, reviewing, or responding to the mail? 
= Ifyes, continue with next screener item. 
«  I/fno, say: Unfortunately, we need to have the conversation with one of the 
people who handles your household’s mail. Can I please speak to one of those 


people? 
« Ifyes and available, repeat the introduction once that person comes to 
the door. 


« Ifyes but not available right now, say: No problem. We can come back 
at a more convenient time for them. What would be a good time for us 
to come back? [record date/time, and, if possible, the other household 
member’s first name to facilitate return visit] Here is a card with more 
information about the study that you can give to them. It also lists our 
contact information in case they would like to contact us sooner about 
participating. [hand Sorry We Missed You card to household member] 
Thank you for your time! Have a nice day. 

« Ifno one else lives here or not interested in you speaking with anyone 
else, say: No problem. Thank you for your time. Have a good day. 


Scheduling 


Great, thank you! You are eligible to participate in the study. We can meet right now if it is 
convenient for you. The conversation will take about 90 minutes. 


e Ifthey are available now, start the interview. 

e /[fthey are not available now, say: No problem. We can come back at a time that is 
more convenient for you. What days this week would be convenient for you to 
meet? 

= Based on your availability, work with the respondent to find a time for the 
interview. 
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Record the date/time for yourself, and on an Appointment card, and say: We 
have you scheduled on [DATE] at [TIME]. Here’s an appointment 
confirmation card for you to keep as a reminder. 

Hand them the card and then say: And could IJ have your first name for my 
appointment calendar? 

Record name, then say: We would like to reach out to you the day before the 
conversation to confirm that this time still works for you. Do you prefer that 
we Call you or that we e-mail you? 

Once they respond, say: And what is the best [phone number/email address] 
for reaching you? If they express concern about providing this information, let 
them know this will only be used to remind them about the appointment and 
confirm they are still available. 

Record answer for your records and say: Thank you for participating! If any 
questions or conflicts come up between now and [DATE OF INTERVIEW], 
please call or email us using the information listed on your appointment card. 
I look forward to seeing you soon. Have a nice day. 
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Materials Used During Interviews 


G.2.1 Informed Consent Form 


mM 


AMERICAN INSTITUTES POR RESEARCH 


Study Participation Consent Form 


The U.S. Department of Education’s National Center for Education Statistics (NCES) is conducting this study to 
understand how to make its research more useful and helpful. NCES has hired American Institutes for Research 
(AIR) to carry out this study. 


We are asking you to participate in a 90-minute conversation where we will ask your opinion about a 
variety of topics, such as how you like to spend your time, how you go about day-to-day activities 
like sorting your mail, and how you feel about participating in research. 


Your participation will contribute valuable insights that will help us improve NCES surveys; however, 
your participation is completely voluntary, and there are no penalties should you choose not to 
participate in this study. You may stop the conversation at any time without penalty. 


The information you share with us will be combined with the responses of other participants in a 
summary report that does notidentify you or anyone else individually. All of the information you 
provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable 
form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). 


If you have any questions or concerns related to your participation in this interview, please contact 
Sarah Grady at the National Center for Education Statistics. You can reach her at 202-245-7063 or 
nhes@ed.gov. 


We would like to record this interview to make sure that we accurately capture the details that you 
provide. However, if you do not agree to be recorded, we will not record the interview. If you agree to 
be recorded, only AIR and NCES employees working on this study will be able to listen to the recording. 
All recordings will be destroyed at the end of the study. Do we have your permission to record the 
interview? 


RIYES FINO 


By signing your name below, you agree to participate in this study. 


Signed: 
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G.2.2 Unstructured Interview Protocol and Demographics Form 


The goal of each interview is to conduct an efficient interview and observation with an 
ethnographic approach in order to glean an in-depth understanding of participants’ 
perspectives and experiences on a variety of topics related to surveys. The interaction should 
be a conversation where the participant does most of the talking and you, through active 
listening and engagement, probe them on particular objectives. Therefore, there is no set 
script. You must rely on your experience and training to emerge from the situation with the 
rich detail needed to address why this particular household did not respond to the National 
Household Education Survey. 


Previous research has elucidated the role of certain factors, such as civic engagement and 
privacy concerns, in why people do not respond to surveys. However, an important 
component of this qualitative research project is to explore the context (physical, social, and 
emotional) in which these nonrespondents are embedded, so that themes can arise from the 
data that help explain why people did not respond to the screener. We are interested in why 
they did not participate, which is likely to be different for different people. For some, they 
may not be interested in taking surveys in general. For others, they may have wanted to 
participate but other factors in their lives intervened. Additionally, some households may 
not have received the initial screener. To accomplish that goal, the interviewer and the 
interview guide need to be flexible and able to explore and probe on issues the participant 
deems relevant. 


1. Before you begin the interview, give the informed consent document to the 
participant, answer any questions they may have, and begin recording with their 
permission. Once they sign the informed consent, put it away in a folder. 


2. Start with rapport building through basic questions about household make-up 
(Domain 1). 


3. Next, proceed with the Experience with Mail Delivery and Experience with NHES 
Mailings and Screener domains (Domains 2 and 3). 


4. After this, you should rely on your training to decide what is the best course forward 
that will yield the most informative interview; however, you should at least touch on 
all of the domains listed in this document. 


5. When there are about 5 minutes left in the interview, ask the questions on the 
demographics form. 


6. Finally, hand the participant their incentive and make sure they sign the incentive 
receipt form. 


Though we generally want to start with the Experience With Mail Delivery and Experience 
With NHES Mailings and Screener domains, we expect that other domains, such as Time Use 
and Privacy Concerns, will arise during these discussions. The domains may flow into one 
another or may not be discrete categories to the participant. For example, you may begin 
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talking about the participant checking her mail, which leads into a conversation about what 
activities she enjoys in her free time. Should that happen, you would want to interweave 
questions about time use and the related objectives. Although the conversation will be free- 
flowing, we want to get a chronological sense of how they handle their mail. For example, do 
they check their mailbox every evening around the same time? Do they review the mail 
immediately? Do they sort the mail while standing at the mailbox? Do they come inside and 
put it away to sort for later? 


The remainder of this document lays out the key domain/objectives to be covered in the 
interview and proposes sample questions and probes you may want to ask in order to fully 


explore that objective. However, the key to this interview is your ability to listen to 


participants’ responses and devise follow-up questions in the moment that serve to deepen 
our understanding of their point of view. The sample questions are examples and may not 


be relevant or meaningful to every participant. Also, itis very important to let the participant 
use their own words to answer questions. Probe without putting words in their mouth with 
very open questions. For instance, ask “Why do you do that?” as opposed to “Do you do that 
so you can X?” 


In addition to understanding the participants’ experiences around the given objectives, each 
interview will include two activities as part of the Experience with Mail Delivery and 
Experience with NHES Mailings and Screener domains. 


1. Mail Delivery: In the first, you will offer the participant a stack of mail and the 
participant will talk aloud about the process of sorting, organizing, and making 
determinations about how they would handle each piece if they received it, and will 
show you where and how that process occurs in the home. 


2. NHES Mailings and Screener: In the second, you will hand the participant the NHES 
materials that were mailed to them earlier this year and the participant will talk aloud 
about his or her reactions to and opinions about the mailings. Finally, you will show 
the participant the NHES screener questionnaire and talk through their opinions and 
reactions. 


Throughout the interview, please incorporate the participant’s surroundings into your 
questioning as appropriate. Ultimately, you want to pull in objects or observations to discuss 


if you believe they may be relevant to why this household did not respond to the screener. 
For example, if you see an American flag on the desk you may want to ask something like: 
“Can you tell me a little bit about the American flag on your desk?” or “I see you have an 
American flag on your desk. What, in particular, is important to you about displaying the 
flag?” The household observation is part of the interview process. You will want to take note 
of their surroundings: what do they hang on their walls? Do you see visible mail piles? Are 
there children’s toys present? What types of books or magazines are displayed? At the 
conclusion of the interview, in your field notes, you will need to write down your 
observations and describe (using thick description as discussed in the training) the exterior 
of the home and the rooms to which you had access during interview. 
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Interview domains 


Domain 1: Rapport Building and Household Make-up 


Objective: To understand key aspects of the participant’s household composition and build 
rapport. 


Sample Questions 
e Tell mea little about who lives here with you. 
e [see that you have three kids. What types of activities do they enjoy? 
e What drew you to the neighborhood? 
e I noticed X (such as a security sign, holiday decor, etc.) outside your home. Can you 
tell me a little about it? 


Domain 2: Experience with Mail Delivery 


Objective: To understand how the household interacts with their mail. For example, how 
mail enters the participant’s household, what happens to it after it arrives, how participants 
decide what mail to keep/open/throw away, and how easy or difficult it is for the household 
to check or send their mail. Have the participant walk you through their mail receipt routine, 
asking questions about timing and activities along the way. During this exchange you will 
incorporate the activity below. 


Sample Questions 

e Where do you get your mail (home delivery mailbox, home delivery mail slot, P.O. 
Box)? Can you show me? (if nearby) 

e What kinds of mail do you typically get? 

e How do you know when mail is delivered? 

e How often do you check your mail? 

e Where do you send your mail (walk to P.O. box, down a building mail chute, mailbox, 
etc.)? Can you show me? (if different from delivery) 

e How do you decide which pieces of mail to open? 

e How would you describe the kinds of mail you keep when you sort through 
unopened mail? 

e How about the kinds of mail that you throw away? 

e What makes you throw away a piece of mail before opening it? 

e What makes you throw away a piece of mail before reading it? 

e What do you do with your mail once you've read it? 

e How do you keep track of mail you need to send/respond to? 

e How often does mail for other addresses get mixed up with your mail? 


Mail Interaction Activity: Hand the participant the stack of mail prepared for this interview. 
Tell the participant to imagine that this was what they received in today’s mail and that their 
name is Jaime Smith and that they live at 123 Main St Anytown USA. Ask them to talk out loud 
as they walk through what they would do with the stack. Participants should verbalize what 
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they are doing at each step. Make sure the participant covers sorting, organizing, distributing 
among household members, opening, storing, saving, discarding, and responding to mail. It is 
important to try to get the participant to verbalize which aspects of each mailing are leading 
them to make various decisions about what to do with them so that we can apply these findings 
to the potential redesign of future NHES mailings. The focus is on the envelope/exterior of the 
mailing and not its contents (many of the envelopes are empty). The prepared stack should 
include one NHES mailing. Observe what the participant does with this envelope during the 
activity, but wait to probe about it until the sorting activity is complete. In other words, let the 
participant treat it like the other pieces of mail. You will only call attention to it at the start of 
the next activity. 


Domain 3: Experience with NHES Mailings and Screener 


Objective: To gather participants’ opinions and reactions to the survey packages that were 
mailed to them earlier this year. To the participant, this should be a seamless transition from 
that activity above. In practice, this is one activity that covers multiple domains. 


NHES Materials Activity: Locate the NHES screener material from the stack of mail and ask 
them to open it and “process” it as they normally would. Observe how they open it. Once it is 
opened, ask participants their opinions and reactions to it. Please cover all aspects of the 
materials - the design, layout, color, wording, the return address, the mailing address, the 
stamp, the envelope size, etc. Probe on what would make participants more likely to open, read, 
and respond to the mailing. 


Sample Questions 
e Do you remember receiving this? (Hand the respondent the mailings (see below for 
the list) one at a time in the order they would have received them. Allow them a 
moment to look at each mailing so that they can answer if they remember receiving it. 
After they say if they remember, follow the general line of questioning below). 
a. Advance letter + envelope 
Initial screener invitation letter + $5 cash + envelope 
Screener pressure-sealed mailer/postcard 
Second screener reminder letter + envelope 
Third screener reminder letter + questionnaire + envelope 
Fourth screener reminder letter + questionnaire + envelope 
e If the participant remembers: (Note to interviewers: Repeat this line of questioning 
for each of the NHES mailings that the participant remembers.) 
o What do you remember about receiving this mailing? 
o What did you do with this envelope when you received it earlier this year? 
o (If opened the mailing): What do you remember about opening the letter? 
What stood out to you? 
o (If did not open the mailing): What changes could we make that would make 
it more likely that you would open this mailing? 
o Why did you throw it away (or keep it)? 
o (Ifremembers a letter that mentions the web screener): Why did you (or did 
you not) access the web page listed in the letter? 


ao om oe 
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= (If accessed the web page): What do you remember about accessing 
the web page listed in the letter? 
= (If accessed the web page): What stopped you from filling out the 
survey? (Confidentiality concerns, technical issues, not interested in the 
topic?) 
(If remembers a mailing that includes a paper questionnaire): What do you 
remember about the paper screener that you received? 
=" What stopped you from filling it out? 
What changes could we make that would make it more likely that you would 
respond to this survey? 


If the participant does not remember any of the materials: 


O 


Oo 0 0 O 


Based on what you see here, do you think you would complete this survey if 
you got it in the mail? Why or why not? 

What stands out to you as you look at it right now? 

If you received this in the mail today, how likely would you be to open it? 
What would make you open (or not open) it? 

What changes could we make that would make it more likely that you would 
open these mailings? 

What changes could we make that would make it more likely that you would 
respond to this survey? 


Domain 4: Understanding of and Attitudes Toward Surveys 


Objective: To gauge the participant’s level of knowledge of and comfort with surveys 


Sample Questions 

What do you think of when I use the word ‘survey’? What do you understand about 
what a survey is? 

Have you ever done a survey before? 

What are the benefits, if any, to participating in a survey? What, in your opinion, are 
the costs/risks/burdens to participating? 

Tell me about the last time you completed a survey. 

Do you enjoy completing surveys? What do you like/not like about doing them? 
How do you like to take surveys? (probe on paper, computer, tablet, phone, in- 
person) 

What are your experiences with getting money or gift cards for completing surveys? 


Domain 5: Privacy Concerns 


Objective: To understand how the participant views privacy and how this may affect their 
willingness to do surveys. 


Sample Questions 
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What does the word “privacy” mean to you? Tell me the first few words that pop 
into your head. 
How do you feel about information that is collected about you and how it is being 
used? How do you think it is used? Does it matter to you who is collecting the 
information? (Note: If the participant is unclear about the meaning of the question, 
you can ask about different settings. “For example, how do you feel about filling out 
forms at the doctor's office? How do you feel about web sites collecting cookies?”) 
How do you feel about the amount of control you have over your personal 
information? 
What are some things you do to protect your personal information? 
How do you feel about providing information on a survey about yourself or your 
family? Does it matter who is doing the survey? Does it matter what the question is 
asking? (If so): What kinds of questions make you uncomfortable? 
Thinking about your daily life, when you have private information you would like to 
share with another trusted person or organization, how secure do you feel sharing it 
through the mail? Online? Via email? Over the phone? What makes you feel secure / 
insecure about information sharing? 
For respondents with children: 
o What rules, if any, do you have about your child/children using the Internet? 
o What discussions have you had with your child/children about giving out 
information when s/he is/are online? 


Domain 6: Attitudes Toward Government 


Objective: To understand the participant’s attitudes toward the federal government and how 
this may affect their willingness to participate in government-sponsored surveys. (Try to 
avoid extended discussion of presidential and electoral politics). 


Sample Questions 


What role do you believe federal government agencies should play in collecting 
information about people through surveys? What information do they need? 

Why does the U.S. continue to do a Census every ten years? What is that information 
for? How do you feel about being asked to complete a Census form? 

What are your thoughts on the Department of Education? What role does it play in 
your life? 

How important is voting to you? 

How much impact do you think the federal government has on your daily life? 
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Domain 7: Education 


Objective: To understand the participant’s opinions about education. 


Sample Questions 
e What role do you think parents should take when it comes to their child’s schooling? 
e In general, what are your thoughts on today’s education system? 
e Are you happy with the school your child attends? Why/why not? 
e How involved are you with your child’s school? 
e How important/valuable was your own education experience to your daily life? 


Domain 8: Time Use 


Objective: To understand how people use their time day-to-day to get a sense of their time 
commitments and priorities. 


Sample Questions 
e Can you briefly describe a typical weekday/weekend day? 
e What activities do you make sure you do every day? 
e How much time do different people in your family typically spend at home? 
e What do you do to relax? 


Domain 9: Civic and Community Engagement 


Objective: To explore the participant’s connection to the community, which has been shown 
to influence survey participation. 


Sample Questions 
e Tell me about any news sources you follow. 
e To what extent to do you feel like an important part of your neighborhood or 
community? 
o What makes you feel like an important part? What makes you feel like not an 
important part? 

e What types of groups are you involved in? 

o What about other members of your family? 
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Domain 10: Household Information and Demographics 


Objective: To understand the types of households and people with whom we are speaking 
and to determine if different subsets of nonrespondents have different types of concerns or 
reasons for nonresponse. 


Lastly, I’ll ask you a few questions about your home and the people who live here. These are 
basic demographic questions like the kinds you are asked on the Census. [Note to 
interviewers: Please read all the questions in this section and ask them exactly as worded.] 


If applicable: We may have already spoken about some of these topics, but I would just like 
to confirm this information now. If you know the participant has already shared this 
information with you, add “Just to confirm” before the question; otherwise, questions should be 
read exactly as shown. 


1. Including yourself, how many people live here? 
[ ] Don’t know 
[ ] Refused 
(if only 1 person lives here, skip to question 2; otherwise, ask question 1a.) 


a. How many people who live here are younger than 21 years old? 
[ ] Don’t know 
[ ] Refused 


2. Whatis your current age? 
[ ] Refused 


3. What is your gender? (Do not read response options.) 
[ ] Male 
[ ] Female 
[ ] Other, specify: 
[ ] Refused 


re you of Hispanic, Latino, or Spanish origin? 
] Yes 

] No 

] Other, specify: 
] Refused 


A 
[ 
[ 
[ 
[ 


5. Whatis your race? (Do not read response options. Select all that apply.) 
American Indian or Alaska Native 

Asian 

Black or African American 

Native Hawaiian or other Pacific Islander 

White 


[ ] 
[ ] 
[ ] 
[ ] 
[ ] 
[ ] Refused 
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Which language is spoken most often by the adults who live here? (Do not read response 
options.) 

[ ] English 

[ ] Spanish 

[ ] Other, specify: 
[ ] Refused 


Do you have Internet access at home...? 

a. Ona computer? []Yes []No  []Don’tknow _ [ ] Refused 
b. How about on a tablet? []Yes []No  []Don’tknow _ [ ] Refused 
c. How about on a cell phone? []Yes []No  []Don’tknow _ [ ] Refused 


What is the highest grade or level of school that you have completed? (Do not read response 
options.) 

[ ] 8 grade or less 

[ ] High school, but no diploma 

[ ] High school diploma or equivalent (GED) 

[ ] Vocational diploma after high school 

[ ] Some college, but no degree 

[ ] Associate’s degree (AA, AS) 

[ ] Bachelor’s degree (BA, BS) 

[ ] Some graduate or professional education, but no degree 
[ ] Master’s degree (MA, MS) 

[ ] Doctorate degree (PhD, EdD) 

[ ] Professional degree (MD, DDS, JD, LLB) 

[ ] Refused 


Are you currently employed and working for pay? 
[ ] Yes 
[ ]N 
[ ] Refused 
e you a full-time or part-time student? 
Yes 


A 
[ 
[ 
[ Reread 


r 
] 
|N 
] 
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11. Based on this sheet [Hand participant the income sheet], which category best fits the total 
income of all persons in your home over the past 12 months? Include your own income. Include 
money from jobs or other earnings, pensions, interest, rent, Social Security payments, and so on. 
(Do not read response options. If participant is struggling, let them know they can make their best 
guess.) 

[ ] A: $0 to $10,000 

[ ] B: $10,001 to $20,000 

[ ] C: $20,001 to $30,000 

[ ] D:$30,001 to $40,000 

[ ] E: $40,001 to $50,000 

[ ] F: $50,001 to $60,000 

[ ] G: $60,001 to $75,000 

[ ] H: $75,001 to $100,000 

[ ] I: $100,001 to $150,000 

[ ] J: $150,001 to $200,000 

[ ] K: $200,001 to $250,000 

[ ] L: $250,001 or more 

[ ] Don’t know 

[ ] Refused 
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co onrtamM obo Oo fF SP 


A 


L 


$0 to $10,000 
$10,001 to $20,000 
$20,001 to $30,000 
$30,001 to $40,000 
$40,001 to $50,000 
$50,001 to $60,000 
$60,001 to $75,000 
$75,001 to $100,000 
$100,001 to $150,000 
$150,001 to $200,000 
$200,001 to $250,000 


$250,001 or more 
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G.2.4 Mail Activity Materials 


U.S. Department of Commerce envelope (NHES initial screener package) 


US DEPARTMENT OF COMMERCE AN EQUAL OPPORTUNITY EMPLOYER wat eee 
Economics and Statistics Administration POSTAGE & FEES PAID 
US Census Bureau raat peaks 
1201 E 10th Street U.S. CENSUS BUREAU 
Jeffersonville IN 47132-0001 PERMIT NO, G-58 


OFFICIAL BUSINESS 
Penalty for Private Use $300 


BC-1325 (5-2011) 


SEQO01-023808 Please respond within two weeks. 


DEO WT kone TT TY oo bb 
MEMBER OF ANYTOWN HOUSEHOLD 

123 MAIN ST 

ANYTOWN, ST 54321-6789 


Cens States™ 


ensus 
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National Council for Education envelope 


National Council for Education 


1770 L St. NW, Suite 800 PLACE 
Washington, DC 20036 STAMP 
HERE 
smith Household 
123 Main St. 


Anytown, st 54321 


USPS National Customer Support Center envelope 


NATIONAL CUSTOMER SUPPORT CENTER 
UNITED STATES POSTAL SERVICE 

6060 PRIMACY PKWY STE 201 

MEMPHIS, TN 38188-0001 


ADDRESS SERVICE REQUESTED 


SMITH HOUSEHOLD OR CURRENT RESIDENT 
123 MAIN ST. 
ANYTOWN, ST 54321 
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PRSRT STD 
US POSTAGE 
PAID 


MEMPHIS, TN 
PERMIT NO. 4732 


National Center for Public Opinion Research envelope 


£ 
: 
i 
i 
H 
i 
a 
i 
i 
j 
} 


PRESS FIRMLY TO SEAL 


PRIORITY’ 
* MAIL * 


DATE OF DELIVERY SPECIFIED* 
(USPS TRACKING™ INCLUDED* 
$ INSURANCE INCLUDED* 


23) PICKUP AVAILABLE 
* Domestic only 


‘WHEN USED INTERNATIONALLY, 
A CUSTOMS DECLARATION 
LABEL MAY BE REQUIRED. 


| QDMA EPI4F July 2013 


Ps00001000014 OD: 12,5 x 9.5 
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PRIORITY MAIL 


PRESS FIRMLY TO SEAL POSTAGE REQUIRED 


National Center for Public Opinion Research 
3823 Anderson Bivd., Suite 419 
Philadelphia, PA 19140 


Anytown Household 
123 Main St. 
Anytown, ST 64321-6789 


USPS TRACKING # 


AMAL 


5 5096 9993 9833 5830 87 


eas) 


ORD 


Service; July 2013; All rights reserved. 


This packaging is the property of the U.S. Postal Service® and Is provided solely for use in sending Priority Mall® shipments. Misuse may be 
violation of federal law. This packaging Is not for resale. EPI4F © U.S. Postal 
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Utility company envelope 


: PRSRT STD 
Gas & Electric Co. US POSTAGE PAID 
Street 


PERMIT NO 35 


RKRKAKAKEH ER 12 O02 451 OPEN IMMEDIATELY 


Current Resident 
123 Main Street 
Anytown, ST 54321 - 6789 


Note: This envelope differed slightly by site, with each site’s envelope including the name and address of a local utility company. 
The name and address have been redacted from the envelope; the city and state have also been redacted from the prepaid postage. 
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Insurance envelope 


PROGRESSIVE INSURANCE 
6300 Wilson Mills Rd. 
Glendale, AZ 85303 


(Untootate 
BUSINESS REPLY MAIL PS ai, 
a ES 
— 


POSTAGE WILL BE PAID BY ADDRESSEE 


Jaime Smith 


123 Main St. 


Anytown, ST 54321 


Quality Bankers envelope 


Q Quality Bankers 
P.O. Box 67104 * Anytown, ST 54321 


% INTRO APR UNTIL $ ANNUAL 
SEPTEMBER 2020 FEE 


Don’t miss out. SAVE NOW 
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PRSRT STD 
US POSTAGE 


PAID 
QUALITY BANK 


weeeeneeee® AUTO SGH 3 DDOT 13M 
JAIME SMITH 

123 MAIN ST, 

ANYTOWN, ST 54321 
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Personal notecard envelope 


\essica Sonche , 
3544 Hoyt Cr. : z 


Anytousn ST 34567 


—laime Sonn 
123 Main s+. 
Anytown, ST 5430 | 
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Candidate forum flyer 


ANYTOWN MIDDLE SCHOOL - GYMNASIUM 
3271 JEFFERSON BLVD, ANYTOWN, ST 54321 


Sponsored & Moderated by: Anytown Chamber of Commerce | anytownchamber.org 


Dentist appointment reminder card 


C904 
DENTIST 
APPOINTMENT 
REMINDER 


June 5, 2019 | 11:15 AM "y 


This is a reminder of your upcoming ¢ 


appointment with 


Dr. Strayer. ey 


Please call our office if you need to 
cancel or reschedule. 


1782 Columbia Rd. Suite 3B 
Anytown, ST 54321 
555-444-3333 


CAa@Y 
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G.2.5 Short Interview Questions 


First, can you please confirm that you are at least 18 years old? 
o Yes (continue) 
o No (go to end interview) 
o Don’t know (go to end interview) 
o Refused (go to end interview 


Can you also confirm that we are at [STREET ADDRESS]? 
o Yes (continue) 
o No (go to end interview) 
o Don’t know (go to end interview) 
o Refused (go to end interview) 


Did you live or stay here most of the time between January and April 2019? 
o Yes (continue) 
o No (go to end interview) 
o Don’t know (go to end interview) 
o Refused (go to end interview) 


Do you handle your home’s mail - for example, do you collect, review, or respond to the 
mail? 

o Yes (continue) 

o No (go to end interview) 

o Don’t know (go to end interview) 

o Refused (go to end interview) 


Have you gotten any mail from the Department of Commerce or Census Bureau since 
January 1st, 2019? 

o Yes (continue) 

o No (continue) 

o Don’t know (continue) 

o Refused (continue) 


How about any mail about the National Household Education Survey? Have you gotten 
any mail like this since January 1st, 2019? 

Yes (ask G) 

No (go to H) 

Don’t know (go to H) 

Refused (go to H) 


oO O00 0 
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G. Would you be willing to share some of the reasons you did not complete this survey? 
(Allow respondent a moment to think.) 


H. Are any of the people who live here age 20 or less? 
o Yes (go to end interview) 
o No (go to end interview) 
o Don’t know (go to end interview) 
o Refused (go to end interview) 


End interview: Those are all my questions. Thank you for your time. Have a nice day! 
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G.3. Qualitative Interview Codebook 
Introduction 


The following codebook represents the first step in analyzing the interviews from the 
National Household Education Survey (NHES) study of nonresponding addresses. The 
codebook contains 15 main codes that capture participants’ experiences and opinions about 
who they are, how they spend their time, and why the household did not respond to 
NHES:2019. The codes are: 


Who’s Who 
Time Use 
Government 
Education 
Surveys 
Technology 
Privacy 
Community 
Life is Hard 

. Beliefs 

. Identity 

. Mail Delivery 

. Example Mail 

. NHES 

15. Memos 


i 


SOP OM ONG OOS 


PRP P 
WNRO 


ran 
iN 


Each main code contains 5-22 subcodes that capture more detailed or nuanced information 
about the broader topic (e.g., “daily routine” subcode within “Time Use”). The codebook also 
contains “adjective codes,” which were always used in conjunction with main codes and 
subcodes to add detail, depth, and context about the discussion (e.g., “would not read” 
adjective code could be used with the “1st mailing” subcode). Each code has a definition, 
exclusion and inclusion criteria, and at least one example. Each subcode includes a definition, 
a mock example used for coder training purposes that illustrates the concept that the 
subcode captures (but does not appear in an actual interview transcript), and/or a sample 
text segment taken from an interview transcript to which the subcode was applied. 


The codes were selected in two stages. In the first stage, the team reviewed the interview 
protocol, which includes the domains and sample questions. Given that senior interviewers 
were instructed to cover all domains in every interview, the team created a set of deductive 
codes that corresponded with each domain—such as privacy, understanding surveys, and 
time use. In the second stage, the team reviewed the memos. Based on this review, the team 
refined the deductive codes and added inductive codes, such as “Life is hard.” 


The codes were applied to segments of text in the interview transcripts. Coders were taught 
to apply a main code or subcode when it best represented the sentiments in that part of the 
discussion. When applicable, coders used a subcode; however, if the discussion did not fit 
under a specific subcode, yet addressed the concepts conveyed in the main code, the relevant 
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main code was applied. Whenever a subcode was applied, the “parent” main code was 
automatically applied as well. For example, if the coder applied the subcode “children” to a 
sentence in a transcript, the parent main code “Who’s Who” was automatically applied to 
that same sentence. 


Importantly, multiple topics often resided in a single sentence or paragraph. For example, 
the sentence “I help my son with his homework” draws on three subcodes “unpaid labor,” 
“homework,” and “children.” Therefore, that sentence would have three subcodes applied to 
it. These subcodes fall under the “Time Use,” “Education” and “Who’s Who” main codes, so 
those three main codes would be applied automatically to the sentence as well. 


After an initial stage of coding of a subset of the interviews, additional subcodes were added 
to the codebook to capture ideas that emerged from the data that were not represented in 
the original codes. Transcripts that had already been coded were revisited to apply these 
new codes. 


Codes are an efficient way to organize massive amounts of textual data and help the team 
conduct analyses but cannot themselves answer research questions. Therefore, after all 
transcripts were coded, we also analyzed the coded transcript data via multiple readings of 
the text. These multiple reads constituted the second step of the analysis. During this step, 
we used the text captured in the codes to identify patterns in the data that answered the 
study’s research questions about survey nonresponse. For example, the subcode “children” 
captures participants’ discussions about children, but the code itself does not tell us what 
they say about children. Only through multiple readings of the text captured under “children” 
could the team explicate the connections between children and nonresponse to the NHES 
screener. 


“Who's Who” Code 


Definition: Any discussion that mentions the people the participant or household members 
interact with now or in the past. Use subcodes when participants mention a certain 
relationship (e.g., children, partner). 


Use only the main only when the type of person the participant talks about interacting with 
does not have its own subcode. Training Example: “! always get my morning coffee from the 
same barista,” or “Yesterday, a salesman stopped by the house.” If the participant refers 
generally to his or her family without mentioning specific family members, use the main 
code. Training Example: | come from a large family.” 


Does not include: 


e Individuals or groups about whom the participant has opinions but no direct 
interaction. Training Example: “Millennials know everything about technology.” 
Instead, use appropriate “Community” subcodes. 

e People who are related to the educational system, such as teachers, principals, or 
school administrators. Training Example: “Our middle school doesn’t have enough 
teachers.” Instead, use “Education” and the appropriate subcode. For interactions 
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with educators, use the Education subcode “interaction with educators.” Training 
Example: “] met with my child’s teacher last month.” 


Subcodes 


current partner An adult with whom the participant is in a romantic relationship. 
May or may not be married. May or may not reside in the same 
household. 


Training Example: “My boyfriend and I met at a local bar.” 


Text Sample: “[Participant’s wife], she’s a stay at home, so she does 
a ton of activities with [the kids].” (7303) 


former partner An adult with whom the participant had a previous relationship. 
May or may not have been married. Former partner may be living 
or deceased. 


Training Example: “My ex-wife just got remarried last month.” 


Text Sample: “My late husband and I were living over in [city 
northwest of urban area in the state].” (4025) 


children People defined by the participant as his or her child or children. 
May be biological, stepchildren, foster children, or adopted. May or 
may not reside in the same household. May or may not be currently 
adults. 


Training Example: “My daughter loves Elmo.” 


Text Sample: “I live with my son. He’s a college graduate. He 
graduated last year, and he transitioned back home.” (5195) 


grandchildren People defined by the participant as his or her grandchild or 
grandchildren. May be biological, step-grandchildren, or adopted. 
May or may not reside in the same household. May or may not be 
currently adults. 


Training Example: “My grandson always goes straight to the 
refrigerator.” 


Text Sample: “My oldest grandchild is [in elementary school] so 
none of them have cell phones yet.” (7523) 


other family Other family members related by marriage, blood, or choice, such 
as aunts, uncles, cousins, nephews, grandparents, in-laws, 
godchildren, etc. May or may not reside in the same household. 


Training Example: “I have seven cousins in my family.” 


Text Sample: “I want to plant stuff; My great auntie, and my 
grandma used to do that.” (7427) 
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neighbors 


People who reside in properties near where the participant and 
household lives or lived. 


Training Example: “The couple that just moved in across the street 
are nice.” 


Text Sample: “I think a neighbor’s son was mugged, but far away.” 
(4541) 


friends 


People whom the participant or members of the household know 
in-person or online where they are mutual acquaintances or 
friends. 


Training Example: “My friend and I go out to dinner once a month.” 


Text Sample: “Yeah, none of my other friends play sports except that 
guy [participant’s friend].” (4153) 


coworkers 


People with whom the participant or members of the household 
share employment. May or may not work at the same physical 
location. 


Training Example: “The boys at the warehouse where I work bring 
their lunch.” 


Text Sample: “We meet sometimes, the girls who I work with.” 
(5429) 


family close 


Participants’ experiences or opinions that they or other household 
members communicate either in-person or by mail, phone or 
electronically with family frequently. Can also be used when 
participants talk about feeling emotionally close with family. 


Training Example: “We have family dinners for all the relatives at 
my house every Sunday.” 


Text Sample: “Yeah. We’re a very close family. We’re so close.” 
(4271) 


family not close 


Participants’ experiences or opinions that they or other household 
members do not communicate in any way with family on a regular 
basis. Can also be used when participants talk about feeling 
emotionally distant from family. 


Training Example: “We see the grandparents at Christmas but 
that’s about it.” 


Text Sample: “Here I have my oldest son who lives in... Where does 
[son] live? Ah, in [a faraway town]. I hardly see him.” (5429) 
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“Time Use” Code 


Definition: General discussion about how participants or household members spend their 


time daily. 


Include: Discussion about a typical weekday or weekend, as well as activities in which they 
would like to engage if they had more time. Training Example: “] like to do five or six 
different things on the weekends.” 


Does not include: Activities of persons others than the participant or household members. 


Subcodes 
paid labor 


Work done for a wage; participants’ or other household members' 
employment. Include any discussion about the participants’ and 
household members’ job or careers. Include discussion about the place 
of labor. May be past or present. Do not include discussions of future 
career ambitions; instead, use “career ambition” under “Beliefs.” 


Training Example: “My son works during the week at a local fast-food 
restaurant.” 


Text Sample: “I had the past week where I had to work Saturday, but 
that’s just because we had crazy orders come in.” (4289) 


unpaid labor 


just for fun 


Chores, errands, childcare, yard work done by participants or 
household members; activities that contribute to the running of the 
household. Unpaid labor are activities that one theoretically could pay 
someone else to do (childcare, tutor, garden, cook, clean, etc.). 


Training Example: “My spouse and I take turns reading the kids 
bedtime stories and doing baths.” 


Text Sample: “Cooking I hate. Yeah, I think it’s a big time-sucker of mine. 
I’m either making breakfast or I’m packing lunches or I’m making dinner 
or I’m shopping for food that they eat in two days.” (5103) 


Any activity in which participants or members of the household engage 
for entertainment or relaxation purposes such as hobbies. 


Training Example: “I love to read mysteries.” 


Text Sample: “Yes, soccer is a passion for my husband... now we’re 
partners and [later this month] there’s another game in [urban city in 
the state], and the whole family is going to soccer.” (5305) 


daily routine 


Activities such as personal hygiene, exercising, or praying/meditating 
that do not relate to running a household. Include only activities that 
are done on a daily or almost daily basis by participants or household 
members. Daily routines are activities that one must engage in 
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personally in order to obtain the benefit such as brushing one’s teeth 
or exercising. They cannot be outsourced like unpaid labor activities. 


Training Example: “I pray every night before I go to bed.” 


Text Sample: “Well, Monday, through Friday I'm getting up in the 
morning, my son’s getting ready for school, brush our teeth, and I take 
him to school.” (7075) 


volunteer 
activities 


Any activity in which participants or members of the household engage 
without pay in order to further a larger civic or personal purpose. 


Training Example: “I am the Girl Scout leader of my daughter’s troop.” 


Text Sample: “Then my sister got me into quilting stocking caps. And 
then I thought—well, because she does it for the homeless shelter—and 
I decided to do it for cancer survivors, or those that are in treatment. And 
I had done maybe several bags of nice soft, no seams caps.” (4025) 


time stamp 


Discussion of what time participants or household members wake up, 
what time they go to bed, how long they spend working, and/or how 
long they spend doing other activities. This subcode applies to any 
discussion that talks about how long an activity takes or the timeframe 
in which it occurs. Participants do not have to say actual time but can 
use descriptors such as “early in the morning” or “late at night.” 


Training Example: “My wife doesn’t get home from work until 8 pm.” 


Text Sample: “And I leave at 8:30 in the morning and I get out. 
Depending on if I have any appointments or anything, I get in about 3:00, 
2:30 or 3:00, and usually somebody's gone and picked the mail up and 
brought it in.” (4167) 


busyness 


schedule erratic 


Discussion of how participants or household members have a lot of 
activities and/or responsibilities. Participants can directly say that 
they are busy or they can say that they have “little time.” Do not include 
interviewers’ assessments from the memos or your own opinion of 
whether the person seems busy; include only participants’ comments 
and their perceptions about being busy. 


Training Example: “I never seem to have enough time to get things 
done.” 


Text Sample: “Any one of us could, but you know we’re so busy, our house 
here is so crazy. I'm the only one who’s up in the morning, obviously here. 
Then they get up around noonish because they work until graveyard.” 
(6535) 


Discussion about how each day is different for participants or 
household members; participant has a hard time articulating what an 
average day looks like. Do not include interviewers’ assessments from 
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the memos or your own opinion of the person’s schedule; include only 
participants’ comments and their perceptions about their schedule. 


Training Example: “Some days I drive for Uber and other days I do pick- 
up work for a local company. My Monday through Friday looks different 
every week.” 


Text Sample: “My schedule changes according to the needs of the 
department. I work atan emergency room. So if there’s a call-off? I might 
have to come in. I typically work evenings, like 3:00 to 11:00, or 3:00 to 
3:00. I could work any shift.” (7015) 


schedule Discussion about how schedule looks similar day-to-day or week-to- 
predictable week for participants or household members; participant describes an 
average day. Do not include interviewers’ assessments from the 
memos or your own opinion of the person’s schedule; include only 
participants’ comments and their perceptions about their schedule. 


Training Example: “I work 8:00 am to 5:30 pm at a hospital six days a 
week.” 


Text Sample: “AA [Alcoholics Anonymous], definitely. During the week, 
Monday through Friday, I’m always up doing something. It usually 
centers around all the stuff 1 do there. When I go back there at 4:00 in the 
afternoon, I don’t just set the chairs up and make the coffee. I set the 
chairs up, make the coffee, and clean all three bathrooms.” (5449) 


alone Discussion where participants talk about a span of time that they 
spend with no people present. They may speak about it as time alone, 
in solitude, or by themselves. Participants may welcome this span of 
time or they may express loneliness. 


Training Example: “I love my commute home because I have time by 
myself.” 


Text Sample: “When I go home, I don’t really have time to be social. So I 
just go to my room and relax. I really need to recharge the batteries. 
Because I leave the house when it’s dark pretty much and I come back 
when it’s dark.” (5685) 


“Government” Code 


Definition: General discussion about the business of operating the United States, states, or 
local municipalities. Training Example: “] don’t pay attention to the government.” 


Do not include: Discussion about the government or politics related to the NHES or 


nonresponse study. Use appropriate NHES codes. Training Example: “] think ‘the 
Department of Commerce’ on the envelope is confusing.” 
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Department of 
Education 


Discussion related to the federal Department of Education. 


Training Example: “The Department of Education is run like a 
business.” 


Text Sample: “I don’t have any more kids in school so [the 
Department of Education] doesn't bring anything to my brain. Maybe 
when [my granddaughter] starts going to school.” (6331) 


United States 
Postal Service 


Discussion related to the United States Postal Service. It can be 
about specific people such as the household’s mail carrier or 
general discussion of the USPS as an institution. 


Training Example: “In 10 years USPS is going to be out of business.” 


Text Sample: “My familiarity with mailing and using the postal 
service, which I do quite frequently because of what I do.” (6535) 


Census 


Discussion related to the Census or U.S Census Bureau. 
Training Example: “I don’t think 2019 is a Census year.” 


Text Sample: “Well, speaking of the Census, that's what I’m telling 
you—that they want to know how many people there are in the 
neighborhood or in the area you live to know if schools are missing 
or what’s missing in the area.” (4131) 


other federal 


Discussion about the federal government that is not covered by 
other subcodes. 


Training Example: “The government doesn’t work for the people.” 


Text Sample: “Government should be there at a federal level to 
maybe protect you from worrying issues, to create the policies for 
trade, stuff like that.” (7329) 


state or local 
government 


voting and 
elections 


Discussion related to past or present local (town, community, 
county, or state) government. 


Training Example: “Our town’s council meets every Tuesday night.” 
Text Sample: “[My father] was a Party ward leader.”(7627) 
Discussion related to participating or declining to participate in 
local and/or national elections. 


Training Example: “It is important to my family that I vote in the 
next election.” 


Text Sample: “Well, voting should be a priority because who is 
elected to the government depends on it.” (4131) 
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government knows | Discussion about what the federal or local government knows 
about individuals. 


Training Example: “The government already has information about 
my family.” 
Text Sample: “Recently, I’ve heard from people that the government 


knows or has shared information between different government 
offices.” (4711) 


government money | Discussion about how the federal or local government spends 
money. Include discussion of taxes in this subcode. 


Training Example: “The government spends way too much. That’s 
why we have a huge deficit.” 


y 


Text Sample: “I absolutely hate and despise any form of taxes.’ 


(5203) 
government Comments that reveal fear or a sense of being persecuted by the 
targeted federal or local government. 


Training Example: “I feel like the government wishes I wasn’t in this 
country.” 


Text Sample: “I personally feel like the government has my phone 
tapped.” (5265) 


role of government | Discussion about what role the government should play in people’s 
lives and/or what the government should and should not be 
responsible for. 


Training Example: “The government should forgive everyone's 
student loans.” 


Text Sample: “Because, you know, cities need a lot of things. 
Sidewalks, the streets. A lot of holes in the streets. Maybe the 
government comes—hey, we need help. The city needs to help fix the 
street, fix this, a lot of things.” (6681) 


“Education” Code 


Definition: General discussion of the opinions of and experiences with education concerning 
the participant, household, or people important to them.!9 Can include thoughts about 
education in general. Training Example: “] am always talking to my neighbors about our 
local schools.” 


19 People who are important to participants and/or household members may be friends, other relatives not living in the household, or 
community members with whom participants feel an affinity. They may also be people who the participant and/or household members 
do not know yet still feel influenced by what happens to them, such as celebrities and subjects of news stories. 
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Do not include: Discussion about education related to NHES or nonresponse study. Use 
appropriate NHES codes. Training Example: “] don’t like that the survey asks what grade 


my child is in.” 


Subcodes 


preschools 


public schools 


Experiences with or opinions about public or private preschool 
or pre-K programs. Can be about schools in the area or in 
general. 


Training Example: “My nephew learned a lot in Head Start.” 
Text Sample: “His preschool is wonderful. His teachers are 
wonderful.” (7779) 

Experiences with or opinions about K-12 public schools. Can 
be about schools in the area or in general. 


Training Example: “Public schools cost the taxpayers a lot of 
money.” 


Text Sample: “I grew up in public schools, and I think they 
prepared me very well for the future, going to college.” (5685) 


private schools 


Experiences with or opinions about K-12 private schools. Can 
be about schools in the area or in general. 


Training Example: “Private schools in this area are not that 
good.” 


Text Sample: “Then I went to a private school, but it was a lot of 
money. And I felt the same way, like the teachers were not 
acclimated to me.” (6587) 


charter schools 


Experiences with or opinions about K-12 charter schools. Can 
be about schools in the area or in general. 


Training Example: “My town experimented with charter schools 
several years ago.” 


Text Sample: “I’ve had a very different experience with 
education. Didn't like the attitude of the school for my son, based 
on the administration. Chose a charter school because of that.” 
(4129) 


bachelor’s or graduate 
degree programs 


Experiences with or opinions about bachelor’s or graduate 
degree programs. Can be about schools in the area or in general. 


Training Example: “I wanted to go back and get my bachelor’s 
degree.” 
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Text Sample: “Well, first of all, I'm the first one to actually 
graduate high school in my family. And then, of course, first one 
to actually go to college, university.” (4655) 


all other 
postsecondary 


Experiences with or opinions about any other postsecondary 
education, such as associate degree programs, certificate 
programs, continuing education courses, etc. Can be about 
schools in the area or in general. 


Training Example: “I am currently studying to be a welder.” 


Text Sample: “And then I went to [technical school], which I got 
my associate’s in something, but it translates over to 
refrigeration. I went to school for refrigeration. A labor trade, I 
guess.” (4153) 


homeschool 


Experiences with or opinions about homeschooling, including 
why they choose/chose to homeschool. 


Training Example: “Homeschooling is important to my 
husband.” 


Text Sample: “The only hesitation that I have with online 
schooling or homeschooling is them not getting the social 
experience with a public school, or even a private school, setting.” 
(4277) 


homework 


Experiences with or opinions about homework. Can be about 
schools in the area or in general. 


Training Example: “I can’t believe that my first grader has so 
much homework.” 


Text Sample: “90% of what they do at home is homework. 
Because they spend—which is my biggest downfall with school in 
general—they spend eight, nine hours a day at school, and they 
come home and do another three or four hours of homework. And 
then they go to bed, rinse and repeat.” (4209) 


K-12 standardized 
tests 


Experiences with or opinions about K-12 standardized tests. 
Can include district or state tests that occur in K-12 or 
Advanced Placement exams (AP). Does not include college 
admissions tests (e.g., PSAT, SAT, ACT, GRE). Can be about 
tests/schools in the area or in general. If someone talks about 
standardized tests or standardized testing and there is no other 
context as to what the specific test is, use this subcode. 


Training Example: “Standardized tests don’t help kids learn.” 


Text Sample: “For me, I have more issues with just generalized 
testing and how education is more focused on the testing aspect 
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of it much more than when we were in schools. I think I have more 
of an issue with that than just town education and all of that.” 


(5103) 
postsecondary Experiences with or opinions about standardized tests taken 
standardized tests for college or graduate school admission or after high school 


graduation. Tests can include the PSAT, SAT, ACT, GRE, MCAT, 
GMAT, LSAT or other professional national tests. Include 
licensing exams such as the bar or NCLEX. 


Training Example: “I did horrible on the SAT so I didn’t get into 
a good college.” 


Text Sample: “But then I had the kids that were getting A's and 
B's in school, couldn't pass the science the first time. Or did not 
score great on the ACT. He didn't take the ACT.” (7471) 


school safety Experiences with or opinions about school safety, such as 
bullying, violence, shootings, or carrying weapons. May be 
general opinions, actual events, fear of events, or actions taken 
to prevent such events (such as metal detectors). Can be about 
schools in the area or in general. 


Training Example: “My nephew was beat up every day by an 
older kid in his homeroom.” 


Text Sample: “When I was at school, none of the doors were 
locked. You just came and went. It was just totally different then. 
You didn't have to worry about, you know, bomb threats and stuff 
like that.” (5449) 


school culture Experiences with or opinions about school environment that do 
not have to do with safety issues. Can be about schools in the 
area or in general. 


Training Example: “Everyone at the school values things other 
than book learning, like being kind and a good friend.” 


Text Sample: “But I always made sure they was in a good school 
that I thought was good for them, and they wore uniforms 
throughout. And that helped out a lot with the peer pressure.” 
(4495) 


afterschool activities Experiences with or opinions about organized activities that 
take place afterschool, such as after-school programs, dance 
classes, Cub Scouts, sports, or music. Can include activities done 
at the school after school hours or activities done at for- or non- 
profit organizations (e.g., dance studio, soup kitchen, 


community pool). Include organized activities that take place in 
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the home, like private music lessons or piano practice. Can be 
about schools /activities in the area or in general. 


Training Example: “My kids take swim lessons on Mondays and 
karate on Wednesdays.” 


Text Sample: “Well, yeah. They love biking and gymnastics and 
soccer and basketball and drums.” (5103) 


subjects 


Experiences with and opinions about what is taught or not 
taught during the school day. Can be about schools in the area 
or in general. 


Training Example: “There is too much focus on math and science 
and not enough on teaching kids how to read well.” 


Text Sample: “Kids know computers now. We don’t need them to 
teach a computer class.” (4639) 


teachers 


Opinions about teachers, their qualifications, their pay, and 
their responsibilities. Can be about teachers in the area or in 
general. Include opinions about teacher quality. For specific 
interactions with teachers, use subcode “interaction with 
educators.” 


Training Example: “The teachers try but they have so many 
things on their plate.” 


Text Sample: “I feel like teachers that graduate aren't held to a 
high enough standard. There’s always a need for them, so they’re 
all going to get hired because they're dressed nice or are nice the 
first couple of days.” (5707) 


interactions with 
educators 


quality of education 


Interactions with people in the education system, such as 
teachers, principals, or school counselors. Any discussion about 
parental involvement in the child’s school (e.g., attending 
parent-teacher or PTA meetings, chaperoning field trips) 
belongs in this code. 


Training Example: “I try to meet my son’s teacher at the 
beginning of the year.” 


Text Sample: “Yes, the teachers call to give us reports, or they call 
to give us some advice about how she’s progressing and how to 
get them to practice more with what they are behind in.” (4017) 


Experiences with or opinions about how well subjects and skills 
are being taught in schools. Can be about schools in the area or 
in general. Can be personal or secondhand experience or 
general thoughts about the issue. 
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Training Example: “We moved to this neighborhood because the 
schools are such high quality.” 


Text Sample: “I don't like the [new] school's curriculum for that. 
It just doesn’t even compare to what he was getting at the other 
school.” (7075) 


importance of 
education 


Experiences with or opinions about how useful and/or valuable 
education is. Can be personal or secondhand experience or 
general thoughts about the issue. Use this subcode when 
participants talk about the benefits of education. 


Training Example: “Giving your child a good education is the 
most important thing.” 


Text Sample: “Well liberal arts to me is important because I think 
when you go to college you need to get into a curriculum or 
agenda where you're interacting with a number of different 
people from a number of different ethnic backgrounds, races.” 
(5463) 


barriers to education 


Experiences with or opinions about obstacles to starting, 
continuing, or finishing an educational degree or credential. 


Training Example: “My son has severe learning disabilities and 
no matter what the district did it didn’t help.” 


Text Sample: “Well, I got pregnant. Then I know I needed 
education, and I needed my schooling. But it was hard for us 
because we were poor. So I had to go to school and work and all 
at the same time. I didn’t get to walk the stage because I was a 
person over forty.” (4043) 


cost of education 


Experiences with or opinions about spending money to pursue 
education, such as student loans, private school tuition for K- 
12, or fees associated with K-12 activities such as field trips. 
Can be personal or secondhand experience or general thoughts 
about the issue. 


Training Example: “I am still paying off my student loans and I 
graduated 10 years ago.” 


Text Sample: “When they got to high school age, we did give 
them a choice [between public and private], because schools were 
pretty expensive.” (7355) 


G-51 


‘Surveys’ Code 


Appendix G. Interview Materials 


Definition: General discussion about surveys and research. Can include opinions about and 
reactions to particular surveys or surveys and research in general. Training Example: “! get 
more and more requests to do surveys.” 


Does not include: Discussion specifically about the NHES or nonresponse study. Use 
appropriate NHES codes. 


Subcodes 


experience with 
surveys 


mode of survey 


sponsor of survey 


Actual experience with surveys of any type, including marketing 
surveys. Also use this subcode when participants say they do not 
do surveys. 


Training Example: “I took a survey a few months ago after I flew 
Delta.” 


Text Sample: “No, I never did a survey, but my daughter did. She did 
a lot with the community. She did. I'm just one of those bored 
widows. I was too busy doing me.” (4043) 


Opinions about the mode of survey contacts (e.g., emails, letters, 
phone calls) or about preferred ways to respond to surveys, such 
as preferring web or paper-based surveys. Include discussions 
about survey platforms, such as doing surveys via Facebook or 
other apps, opt-in panels, etc. 


Training Example: “Paper-based surveys are easier to read.” 


Text Sample: “Well, I think it would be the regular written form 
where I can read, where I can see clearly. Because I don’t have 
Internet there. No, no. If you don’t have access to the Internet or if 
you don't know if the page is correct or, what’s its name, well, yes, 
it’s better with a form.” (4017) 


Opinions about who is sponsoring or conducting the survey (such 
as the government, a market research firm, or a foundation) and 
how that influences participation. 


Training Example: “I don’t do any surveys from the YMCA.” 


Text Sample: “Like if an ad agency called me and asked me to do a 
survey or something, and it’s about personal information, I wouldn't 
do it.” (5441) 


topic of survey 


Opinions about the topic of the survey (customer satisfaction, 
politics, local issues) and how that influences participation. 


Training Example: “It’s important to do surveys about my town 
and how we should spend our surplus.” 
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why complete 


Text Sample: “I think in the past year, I got a food survey. And that 
was important because they were talking about quality of food and 
quality of vegetables and fruits. So, that was important because it 
kind of affected the way we eat.” (4399) 


Reasons why participant would or has completed surveys. Can 
also be used when participant speculates on why other people 
complete surveys. 


Training Example: “I feel like I am giving back to my community 
when I take a survey.” 


Text Sample: “/ did one last week when I went to [a fast-food 
restaurant]. It took five minutes, so that’s why I was like, ‘Hell yeah, 
I'll take a $50 gift card for five minutes.” (4289) 


why not complete 


Reasons why participant would not or has not completed surveys. 
Can also be used when participants speculates on why other 
people do not do surveys. 


Training Example: “I don’t do surveys anymore because they 
always take too long, and they ask the same question over and over.” 


Text Sample: “Millennials and people like that don’t really want to 
participate in a lot of things. They don’t want to deal with door-to- 
door solicitation. Which, we just thought that was interesting, 
because you know with my parents, they’re different that way.” 
(4277) 


benefits of surveys 


Opinions about ways surveys are useful or helpful that are not 
described as a reason why someone would or not would not do a 
survey. 


Training Example: “Surveys can help others learn about what the 
greatest needs are.” 


Text Sample: “Like the Census? Yes? I think it is beneficial, because 
we should be counted, right? The Census is for them to know how 
many of us live here, isn’t it?” (4187) 


drawbacks of 
surveys 


Opinions about ways surveys are problematic that are not 
described as a reason why someone would or not would not do a 
survey. 


Training Example: “Surveys are a waste of time.” 


Text Sample: “It’s a good and bad thing. Well, I mean, they trying to 
receive what’s going on, that’s the good thing about it. But once they 
do see what’s going on, I don’t feel like it’s too much of an effect on 
the community.” (7057) 
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incentives 


Opinions about the use of incentives to encourage people to 
participate in research. Do not include discussion about the $5 
received with the NHES; use the “five-dollar incentive” subcode for 
those discussions. 


Training Example: “Getting a gift card encourages me to do the 
survey.” 


Text Sample: “Because it seems like every time you be like, ‘Hey we'll 
give you a $25 gift card, just answer these couple questions.’ I’m like, 
‘All right, I’m game. I'll do it.’”” (5703) 


“Technology” Code 


Definition: General discussion of technology and how participants, household members or 
people important to them use it. Training Example: “I remember when nobody had a 


computer.” 
Subcodes 
Amazon Experiences with and opinions about any service or product from 
Amazon, such as Amazon.com, Alexa, or Echo. 
Training Example: “I buy everything on Amazon.” 
Text Sample: “Amazon has better data on the populous than the 
government does at this point because they mine data.” (7303) 
Google Experiences with and opinions about any technology from Google, 


such as Gmail, Google Home, Nest, Google maps, or the search 
engine function. 


Training Example: “I always Google someone before I go on a date.” 


Text Sample: “I use Google News. It corroborates all of my topics.” 
(5479) 


tech devices 


Internet 


Experiences with and opinions about the Internet or online 


games. Do not include online financial activities or social media use. 


Experiences with and opinions about tech devices, such as 
computers, laptops, tablets, or cell phones. 


Training Example: “I rarely use my computer; I do everything on my 
smart phone.” 


Text Sample: “Everybody gets a Chromebook, and they bring it 
home, back and forth with them to school. They submit an assignment 
and it just goes, it emails it right to the teacher.” (5103) 


activities, such as emailing, visiting websites, or playing multi-user 
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Training Example: “I spend way too much time online.” 


Text Sample: “I watch a lot of sermons online, YouTube.” (6535) 


money online 


social media 


Experiences with and opinions about online financial activities, 
such as banking and bill paying. 


Training Example: “I don’t trust putting my bank account number 
online.” 


Text Sample: “I do all my banking and have for years online.” (4025) 
Experiences with and opinions about Facebook, Snapchat, 
Instagram, Twitter, and other social media outlets. 


Training Example: “I keep in touch with my family through 
Facebook.” 


Text Sample: “Facebook, I have a site so like if somebody wanted to 
be able to find me, they could. But I have not updated in [several 
years].” (7055) 


“Privacy” Code 


Definition: General discussion about the opinions and experiences with privacy by the 
participants, household members or people important to them. Training Example: “People 
should care more about their privacy.” 


Subcodes 


data security 


Experiences with and opinions about how personal information is 
or is not protected. Include discussion of identity theft or hacking 
and any discussion about online security including passwords. 


Training Example: “It was a nightmare when someone stole my 
credit card information.” 


Text Sample: “I have identity theft protection. That’s about all I got.” 
(5203) 


keep to themselves 


Any discussion about maintaining physical distance from others or 
thinking that it is important/useful to do so. 


Training Example: “Good fences make good neighbors.” 


Text Sample: “I leave the bedroom door shut. That means you don’t 
come in unless you knock.” (7597) 


nobody's business 


Any discussion about maintaining personal and/or 
psychological distance from others or thinking that it is 
important/useful to do so. 
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Training Example: “People don’t need to know my inner most 
thoughts.” 


Text Sample: “Privacy means solitude of your own matters, to me.” 
(5265) 


security Experiences with and opinions about taking steps to address or 
precautions maintain the physical safety of themselves, other people, or a 
physical dwelling. 


Training Example: “I make sure I put security signs in the yard so 
people can see them.” 


Text Sample: “I’ve been thinking of getting the Ring doorbell 
installed here so! can keep a better eye on the neighborhood.” (6221) 


access to Experience with and opinions about how personal information can 
information or cannot be accessed by corporations and nongovernment 
organizations. For talk about government access to data, use 
“government knows” subcode instead. 


Training Example: “Kroger knows exactly what I buy my family 
every week.” 


Text Sample: “I think anyone can really buy out our information. My 
information is through my employer, my information is through my 
school.” (4639) 


protect Experiences with and opinions about taking steps (or steps that 
information one should take) to protect or secure digital or paper records that 
have personal information. Do not include how people get rid of 
mail. Instead, use “mail rid” subcode. 


Training Example: “I change my passwords once a month.” 


Text Sample: “I always keep the state-of-the-art Norton stuff.” 
(7627) 


not a concern Any discussion about privacy not being a concern. 
Training Example: “I’m not worried about my privacy at all.” 


Text Sample: “I’m not really worried about my information getting 
out there. I have nothing to hide.” (5479) 


‘Community’ Code 


Definition: General discussion about how the participant, household members or other 
people important to them interact with and perceive the environment outside the household. 
The goal of the code is to understand how people define community, their opinions and 
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experiences within communities, and how connected, if at all, they feel to any community. 
Training Example: “It’s important to be part of the community.” 


Does not include: Any discussion of specific activities, such as the participant’s job or 
hobbies. Use the “Time Use” code and subcodes. 


Subcodes 


nature Description of or opinions about outdoor areas with any type of 
vegetation. Also include discussions of "peace and quiet." 


Training Example: “I wanted to live in a place with trees.” 


Text Sample: “T'll sit on my patio and listen to a tape or read. I just 
sit down and enjoy nature.” (5409) 


general town Discussion of the area in which the participant and household 
discussion members live or lived. This subcode includes how long they have 
lived in the area and discussions of how that area has changed over 
time. 


Training Example: “Things in this town have changed a lot over the 
last 20 years.” 


Text Sample: “We didn’t consciously say, ‘Oh, I want to live in [city 
1], or [city 2] or [city 3].’ That’s... the job was here, and we needed to 


be close to our jobs.” (4167) 


others' Discussion of the neighborhoods, town or communities of people 
communities who are not the participant or household members. 


Training Example: “Down of here is a suburb that is growing really 
fast.” 


Text Sample: “I care about the Third World countries and stuff over 
there.” (7401) 


news Discussion of methods of receiving information about current 
events. 


Training Example: “I used to listen to NPR, but now I watch NBC 
Nightly News.” 


Text Sample: “I get the news on my phone. So, I do that, and then too, 
I watch the news. In fact, that’s what I was doing too, besides working 
when you guys came. So I always watch the 5:30 national news.” 
(4373) 


what community is | Discussion of how they define community and what community 
means to them. 


Training Example: “Community doesn’t have to be people you see 
every day. It can be people you feel a connection with no matter 


where you are.” 
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Text Sample: “When I think community, I think everyone from our 
apartment complex meeting at the pool and having a barbecue.” 
(4209) 


sense of belonging 


Discussion of feeling connected to members of a certain group or 
groups; an affiliation with others. 


Training Example: “I feel very connected to my bingo group.” 


Text Sample: “Since I'm not from here, I’ve made a lot of friends, 
which is really nice. [The co-op], they do a good job of putting 
everyone together.” (7303) 


outsider 


Discussion of feeling disconnected from other people or groups. 
Include discussions about feeling judged by others or feeling alone 
or lonely. 


Training Example: “I don’t really relate to anyone my age.” 


Text Sample: “I’m just not that much of a social mom person, so I 
really couldn’t take some of the stuff there. Just couldn’t do it 
anymore.” (5103) 


neighborhood safe 


Discussion feeling safe at home and/or in the neighborhood or that 
the neighborhood experiences little to no crime. Can be used for 
current or past neighborhoods. 


Training Example: “I don’t even lock my doors. The neighborhood is 
so safe.” 


Text Sample: “I think everybody on this street, they pretty much look 
out for each other and watch out for each other.” (4399) 


neighborhood 
unsafe 


Discussion of feeling unsafe at home and/or in the neighborhood 
or that the neighborhood experiences crime. Can be used for 
current or past neighborhoods. 


Training Example: “Just yesterday there was a drive-by.” 


Text Sample: “Nobody knocks on my door. Everybody else, they call 
if anybody is coming. ‘Cause this is a dangerous area.” (4495) 


segregation 


Discussion of observing themselves or other people of a certain 
race, ethnicity and/or nationality living in a defined area and/or 
not frequently interacting with people of other races and 
ethnicities. Also include general opinions that are pro-segregation. 


Training Example: “Only White people live in that neighborhood.” 


Text Sample: “It is very siloed. In the [area of town], it’s 
predominantly Puerto Rican. And the [other area of town], Italian. Up 
the hill, African-Americans. So, that’s very reductive, but that's where 
it is.” (5195) 
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diversity 


Discussion of observing themselves or other people of varied races, 
ethnicities and/or nationalities frequently interacting with people 
of other races, ethnicities, and/or nationalities. Do not include 
opinions about diversity, only include experiences; code opinions 
under the “diversity/inclusion” subcode. 


Training Example: “I am glad my child has friends of different 
races.” 


Text Sample: “Ironically, that a city that is perceived as so 
conservative, they've been so okay with gay lifestyles.” (7149) 


SES 


Discussion of socio-economic status or class. 


Training Example: “Since the plant closed, the surrounding 
neighborhoods have gotten a lot worse off.” 


Text Sample: “You could say that I live in a lower sect of society, but 
I enjoy the people I’m around. Life is way, way bigger than putting a 
money value on something. You can have all the money in the world 
and still be a really unhappy person.” (4289) 


“Life Is Hard” Code 


Definition: General discussion about barriers and obstacles participants, household 
members or people important to them face or have faced in life and their reactions to them. 
Training Example: “{ had it really rough growing up.” 


Subcodes 


money concerns 


Discussion of current or past financial burdens, concerns about 
paying bills, debt, making repairs, buying needed items, saving for 
college or retirement, or lower-than-desired wages. Include worry 
about future financial burdens. 


Training Example: “My credit card debit is over $10,000.” 


Text Sample: “My parents did not have the resources to give us an 
allowance and even like save for college and everything.” (5463) 


money helps 


Discussion of how money can increase one’s sense of security, bring 
pleasure, or mitigate life’s difficulties. 


Training Example: “I make 120k a year so a car repair bill doesn’t 
throw me off.” 


Text Sample: “Then I went from being caregiver to office worker. I'd 
make that little extra money.” (7779) 


incarceration 


Discussion of time spent in jail or prison; discussion of arrests and 
court appearances. 


G-59 


Appendix G. Interview Materials 


Training Example: “I was arrested for armed robbery.” 


Text Sample: “I did [several years of time] in prison for a violent 
crime.” (6535) 


police 


Discussion of interactions with or observations of the police. Can 
be positive, negative, or neutral. Can be general opinions about the 
police or specific experiences. 


Training Example: “When we called 911 it took the police over an 
hour to come to the house.” 


Text Sample: “The news makes me aware and probably makes me 
overprotective, because police can’t protect you.” (7075) 


medical conditions 


Discussion of current or past, short- or long-term physical or 
mental health afflictions. 


Training Example: “My son has ADHD.” 


Text Sample: “She has a condition that is difficult for her. So, it’s like... 
it’s some kind of autism, and then it’s hard for her to concentrate.” 
(5429) 


medical treatment 


Discussion of treating or attempting to treat physical or mental 
health afflictions, now or in the past. 


Training Example: “She needs to take four different pills.” 


Text Sample: “I fell and took the skin off and then a week and a half 
later it becomes blood infected, turns red. So then I’m on antibiotics.” 
(4025) 


illegal drug use 


Discussion of taking, selling, or buying illegal drugs, now or in the 
past. Do not include marijuana or CBD oil products. 


Training Example: “I was busted for coke.” 


Training Example: I think it’s just once the drugs got involved the 
neighborhood lost a lot of [safety].” (7471) 


marijuana and 
alcohol 


unemployment 


Discussion of using marijuana, CBD oil, or alcohol. 

Training Example: “I smoke pot to relax.” 

Text Sample: “He went to [other public middle/high school]. He got 
kicked out due to having marijuana on him.” (6081) 


Discussion of losing a job or not being able to secure a job. 


Training Example: “I couldn't find a job after the layoffs.” 
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multiple jobs 


Text Sample: “Right now I was trying to look for work again since 
I’ve been out of work for a while.” (7481) 


Discussion of holding multiple jobs simultaneously. 


Training Example: “My parents work at the restaurant during the 
day and at night they clean offices.” 


Text Sample: “Mondays, Wednesdays, and Fridays, typically I’m at 
my [personal care assistant] job. And generally, Tuesdays and 
Thursdays, I’m at [another job].” (5265) 


death or dying 


exhaustion 


Discussion of any person or pet that has passed or is near death. 


Training Example: “Once my mother died, my dad came to live with 


y 


US. 


Text Sample: “Before I lost my daughter, three months before that, I 
lost a sister.” (4043) 


Discussion of exhaustion, stress, burnout, or feeling down that is 
being experienced or has been experienced by the participant, 
household members, or persons important to that household. 
When depression or anxiety has been diagnosed by a health 
professional, use subcode “medical conditions” instead. 


Training Example: “I travel so much for work that I am too tired to 
look for another job.” 


Text Sample: “I work full time and go to school full time. So, I’m like 
always tired.” (4639) 


anger 


Anger, exasperation, impatience, or resentment expressed by 
participant, household members, or persons important to them. 


Training Example: “The way the youth today treat their elders 
makes me sick.” 


Text Sample: “Anger’s a nice fire and it motivates me.” (7627) 


trouble reading 


Discussion of difficulties with reading and/or writing in any 
language (e.g., due to low literacy, low English proficiency, or poor 
eyesight). 


Training Example: “I like the pictures because I can’t read.” 


Text Sample: “I see that as blue, but my eyes are really bad.” (6287) 


resilience 


Discussion of being able to recover from a setback. 


Training Example: “I had enough grit to raise three kids on my 


y 


own. 
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Text Sample: “I used to just walk out of class. There was never a day 
I was not in the office, never a freaking day, never a freaking day not 
in the [principal's] office. But I finished [school].” (6081) 


life is good 


Discussion of how life is going well for participants, household 
members, or persons important to them. 


Training Example: “I am really happy with the way things are going 
right now.” 


Text Sample: “Here in America, they feed them in school, they do 
everything... In fact, it's just like a dream come true.” (4705) 


“Beliefs” Code 


Definition: General discussion about what participants, household members, or people 
important to them believe or value. Can also be about the underlying principles that 
influence how they prioritize their time. Training Example: “Fighting climate change is 
really important to me.” 


Does not include: General thoughts about education or government, which will be captured 
in the “education” and “government” codes. Training Example: “Education is important; 
“There are three branches of government.” 


Subcodes 


religion 


Discussion of attendance at places of worship. 


Training Example: “I was raised Catholic, but I stopped attending 
church in college.” 


Text Sample: “My daughter and her family go to a church that meets 
on Friday night.” (4025) 


diversity /inclusion 


Discussion of believing that it is beneficial to be inclusive of 
individuals representing more than one national origin, color, 
religion, socioeconomic stratum, sexual orientation. 


Training Example: “Everyone benefits when you have people from 
different backgrounds working with each other.” 


Text Sample: “It has people from all the neighborhoods. You see all 
the colors, smells, and tastes there. All the cultures. Yes, I love it.” 
(5305) 


prejudice 


Discussion of instances of prejudice, discrimination, or bigotry 
(such as racism, sexism, homophobia, religious intolerance, etc.). 


Training Example: “People boycotted when a new mosque was 
built.” 
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patriotism 


Text Sample: “You're assuming just because one person of color acts 
a certain way, or looks a certain way, they all are like that.” (7075) 


Discussion of loyalty to or support of the United States as a nation; 
support for ideals of the United States no matter how those ideals 
are defined. 


Training Example: “Serving your country is the highest calling 
anyone can have.” 


Text Sample: “I’m in America, one of the best countries in the world. 
I feel very blessed, and I could be somewhere worse than here.” (6221) 


military 


trust others 


Experience with or opinions about the military or military service 
(in the Army, Navy, Air Force, Armed Guards, etc.). Can also refer to 
general talk about military forces. 


Training Example: “All of my cousins went into the Army.” 

Text Sample: “You have to bring a diploma in here in order to stay 
here, so she graduated and went into the military.” (7231) 
Expressing confidence or hope in other people, organizations, or 
systems. 


Training Example: “The police do everything they can to protect this 
community.” 


Text Sample: “I think [the news outlets are] telling the truth.” (7481) 


distrust others 


Expressing skepticism, doubt or suspicion about other people, 
organizations, or systems. Use this subcode when people talk about 
scams or suspect that something is a scam. 


Training Example: “The government just takes your money and 
doesn't give you anything in return.” 


Text Sample: “Don’t make it a point to complete government 
surveys... I don’t trust them.” (4365) 


distorted info 


Belief that information including research can easily be distorted 
or manipulated to further an individual or an organization’s 
agenda. 


Training Example: “People manipulate information so it supports 
their own point of view.” 


Text Sample: “As soon as I start to hear the big party rhetoric on 
either side, I’m like, I could never watch CNN, and I could never watch 
Fox. Because it’s all rhetoric.” (7055) 
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minimalism Deemphasizing possessions or belongings; decluttering. 
Training Example: “I want to have less stuff: The less you have, the 
less you have to worry about.” 
Text Sample: “My dream is to go to an island and live on a beach in 
some little rundown little hut.” (4289) 
career ambition Discussion of long term or future career goals and aspirations. Do 
not include work activities that will occur within a year or two, such 
as promotion expected to occur in 2020. 
Training Example: “My daughter wants to be a doctor when she 
grows up.” 
Text Sample: “I need my GED so I can go to hair school.” (7275) 
local not global Discussions that reveal the sense that a person can affect things on 


a local scale but not on a national or larger one. 


Training Example: “Protesting with my neighbors about the new 
road was important but that doesn’t mean the state capital cares 
about us.” 


Text Sample: “I mean local, I think that [voting] matters. But 
nationally, I don’t think so.” (5441) 


what’s wrong 


Discussions that reveal what issues or policies they believe are 
hurting some aspect of the life in the U.S. 


Training Example: “Everything is so expensive these days.” 


Text Sample: “I don’t believe that our votes [matter]—like a 
democracy is not as pure as people try to make it seem.” (4639) 


“Identity” Code 


Definition: General discussion about demographics that are salient to participants, 
household members, or people important to them. It does not have to be discussion about 
the participants’ personal identity. Training Example: “Being from the Midwest is important 


to me.” 


Subcodes 


race/ethnicity 


Experiences related to and opinions about race (such as Asian or 
African American) or ethnicity (such as Mexican or Hispanic). 
Topics include Black Lives Matters, pride in one’s race, 
experiencing racism or oppression or microaggressions. Does not 
have to be limited to the participant’s own race or ethnicity. 
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Training Example: “People should take the Black Lives Matter 
movement more seriously.” 


Text Sample: “I went through my stages of—I was ashamed of being 
Hispanic at one time—so I would tell people that I’m half White, you 
know.” (6221) 


immigration 


Discussion of the immigration status or migration of the 
participant, household members or persons important to them. Do 
not include opinions about whether immigration is a good or bad 
thing. 


Training Example: “My family is from Guyana.” 


Text Sample: “There are a lot of people who don’t have papers, so 
they think that giving [the government survey] that kind of 
information is telling Immigration.” (4711) 


citizenship 


Discussion of what it means to be a U.S. citizen or how the 
citizenship status of the participant, household members or 
persons important to them affect daily life. 


Training Example: “My mom’s goal was for us to be U.S. citizens.” 


Text Sample: “We are citizens. We do not worry about the letter 
[from the government].” (6487) 


dual identity 


Discussion of what it means to be a dual citizen, being raised outside 
the U.S., or having family abroad (as in their country of origin) in the 
context of the participant feeling “attached” to another 
country/culture. 


Training Example: “I try to listen to Caracas radio so that I know 
what’s happening back home.” 


Text Sample: “Those four cultures and languages live in my body and 
my imagination. Does it make me less American? No.” (5195) 


language barriers 


Discussion around having language barriers that make it difficult to 
understand English. Use this subcode for discussions about 
translating for others in household. 


Training Example: “My parents don’t speak English very well, so I 
have to go to all their doctor appointments.” 


Text Sample: “[My son] is learning a lot of English. Yes, we know it 
and we understand it, but we don't speak it and his sisters do.” (4017) 
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“Mail Delivery” Code 


Definition: General discussion about participants’ and households’ experience with mail 
delivery. Discussions about both receiving mail and sending mail. Training Example: 
“Getting actual things in the mail is not as important to me as it once was.” 


Does not include: Do not use with Example Mail or NHES mail activity discussions, unless 
the participant makes a side comment during the activities that is more generally about mail 
delivery. Instead, use “Example Mail” and “NHES” main and subcodes. 


Subcodes 


location of 
mailbox 


location of mail 


Physical location where the participant and household receives 
and/or sends mail. 


Training Example: “Our mailbox is at the end of the driveway.” 
Text Sample: “You come out in the park and as soon as you come in 
the gate, the boxes are right there in a gated area.” (4043) 

Physical location or locations where the participant and household 
members keep or save their mail. 

Training Example: “All mail goes directly on the kitchen counter.” 


Text Sample: “We have a box in the kitchen where we put our bills.” 
(7075) 


mail processing 


Discussion related to how the participant and other household 
members sort or process mail, including what to keep, what to open 
and what to read. 


Training Example: “I look at it really quickly at the mailbox, but I 
don’t make any decisions. I wait until 1am in the house and then I look 
at the return address.” 


Text Sample: “If it’s not a bill, it gets thrown on the kitchen table. And 
if it looks interesting it stays on the kitchen table. If it looks like it’s 
ads, I toss it.” (7779) 


mail rid - toss 


Discussion about throwing away mail. Do not include discussion of 
shredding mail; code that under “mail rid—shred.” 


Training Example: “I just throw away every catalog I get the minute 
I get it.” 


Text Sample: “I’m going to bring this [advertisement] in, and then I'll 
literally walk over to the garbage and throw out.” (5203) 


mail rid - shred 


Discussion about shredding mail (using scissors, hands, or a 
machine). 
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Training Example: “I shred everything I get, especially those credit 
card offers.” 


Text Sample: “If it’s something like a credit card offer, I'll either shred 
it in the shredder machine, or I will take the scissors, and the little fake 
credit cards that come in those offers, I'll cut them up.” (4399) 


mail rid - other 


Discussion about getting rid of mail that does not involve shredding 
or tossing. 


Training Example: You can't be too careful, so I throw all my mail 
into the fireplace and burn it.” 


Text Sample: “[I use a] plastic trash can with bleach, it erases 
everything on the paper.” (7471) 


mail frequency 


Discussion of how often participant and household check the mail. 
Include discussion of the time of delivery. 


Training Example: “I can go days without checking the mail.” 


Text Sample: “I check my mailbox once in two weeks.” (4495) 


trouble with mail 


Discussion related to problems with mail delivery such as 
misdelivered mail. 


Training Example: “We often end up with our neighbor’s mail.” 


Text Sample: “They delivered wrong, so I always used to get it and 
take it to [my neighbors], and be like, ‘Hey, I got your mail at my house, 
and here is your mail.’ (6081) 


typical mail 


Discussion of the typical types of mail the participant receives. If the 
participants go through their own mail, use this code. 


Training Example: “I usually get two catalogs a week.” 


Text Sample: “[I get] my light bill. Sometimes surveys like that. I don’t 
really get mail. I guess I'm not a very important person.” (5757) 


mail carrier 


Discussion of interactions with or perceptions of the person who 
delivers (or delivered) the household mail. 


Training Example: “Our old mailman used to carry dog treats.” 


Text Sample: “Our mail carrier, he’s a cool guy. If he sees the door 
open, he'll ask me if I got anything. Or I'll see him coming and give it 
to him.” (5703) 
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“Example Mail” Code 


Definition: Discussion of the example mail activity. Include participants’ reactions to all 
aspects of the mailing and reasons why they would or would not open it. Training Example: 
“This stack is a lot more mail than I usually get every day.” 


Subcodes 


stamp 


Discussion about the stamp or metered type. 


Text Sample: “This is probably more important because there's a 
regular stamp.” (5441) 


return address 


Discussion the sender and the return address displayed on the 
mailing. 


Text Sample: “Occasionally I would just glance at the return address.” 
(7149) 


recipient name Discussion about using the name “Jamie Smith” or “household...” 
Text Sample: “If it’s my name, I'll open it.” (4187) 

size Discussion about the mailing’s size. 
Text Sample: “If this was a Manila envelope or a larger Priority Mail, 
I would open it, probably.” (5203) 

other envelope Any other comments about the mailing’s design, such as the color of 

design the envelope or additional text like “Open Immediately.” 
Text Sample: “No annual fee tells me, immediately, this is an offer for 
a loan or credit card.” (7149) 

Priority Mail Discussion about a mailing being sent via priority mail delivery. 
Text Sample: “Anything Priority Mail, I know I will open it.” (4705) 

Quality Bankers Discussion about the Quality Bankers mailing. 


Text Sample: “I already have a bank account, so I don't need that.” 
(7503) 


National Customer 
Support Center 


Discussion about the National Customer Support Center mailing. 


Text Sample: “National Customer Support Center... Toss that, because 
I don't know what it's about.” (7401) 


U.S. Department of 
Commerce 


Discussion about the U.S. Department of Commerce mailing. 


Text Sample: “Commerce. Economics. Administration. Yeah. This [is 
the] U.S. Census. I don't want to do that.” (5409) 
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Personal note Discussion about the personal note card. 

card Text Sample: “It’s supposed to be a real card, I would keep it.” (5103) 

Utility company Discussion about the utility company mailing. 
Text Sample: “This is not my energy company, so I don’t need them.” 
(7321) 

Insurance Discussion about the insurance mailing. 
Text Sample: “Maybe I would have looked at this if I was considering 
insurance, but I just happened to get brand new insurance not a week 
ago.” (7015) 

Dentist card Discussion about the dentist appointment reminder card. 
Text Sample: “I have an appointment, so I’m going to stick that on my 
fridge, because that’s important.” (7779) 

Candidate forum Discussion about the candidate forum mailing. 


Text Sample: “I can see that it’s probably about politics here in the 
city I live in.” (4399) 


National Center 
for Public Opinion 
Research 


Discussion about the National Center for Public Opinion Research 
mailing. 


Text Sample: “National Center for Public Opinion. Hmm. I'd still open 
it, because somebody probably spent three to four bucks to send me 
this.” (5203) 


National Council 


Discussion about the National Council for Education mailing. 


for Education Text Sample: “This one it says National Council of Education and 
maybe it has to do with the university.” (4655) 
Mail order catalog | Discussion about the mail order catalog mailing. 


Text Sample: “And [this store], yeah that’s definitely not for me.” 
(7329) 


Box store circular 


Discussion about the box store circular mailing. 


Text Sample: “I don’t shop generally at [this store]. Unless there’s 
something on the front that catches my eye, it’s put in the trash mail.” 
(4167) 
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Definition: Any discussion related to the National Household Education Survey and related 
materials and the nonresponse study. Training Example: | remember getting the NHES 


letter.” 


Subcodes 


1st mailing 


Any discussion about the initial NHES screener mailing including, 
remembering receiving it (or not), the appearance, ease or 
difficulty opening it, and mailer contents. 


Training Example: “I asked my neighbor if they got this letter.” 


Text Sample: “It [first mailing] is from the United States Department 
of Commerce. That means it's something to do with us in America.” 
(4043) 


pressure sealed 


24d mailing 


Any discussion about the pressure sealed envelope, including 
remembering receiving it (or not), the appearance, ease or 
difficulty opening it, and mailer contents. 


Training Example: “This is hard to open.” 


Text Sample: “I said, okay, if they’re sending me this for a second 
time, then I need to do it.” (7553) 


Any discussion about the 2"4 NHES screener mailing, including 
remembering receiving it (or not), the envelope and the contents 
or appearance of the letter. 


Training Example: “I don’t like the tone of the language in this 
letter.” 


Text Sample: “At this point, if this were the third one I've gotten, it 
probably would go directly into the shred without me doing another 
thing.” (4167) 


3'4 mailing 


Any discussion about the 3™ NHES screener mailing, including 
remembering receiving it (or not), the envelope and the contents 
or appearance of the letter. 


Training Example: “I remember setting this aside to go to the 
website.” 


Text Sample: “But if it didn’t have my name, at this point [after three 
earlier mailings], out of the envelope, unfolded, in my face, even with 
this barcode even with the United States the Department of 
Commerce, even with all that, shred.” (6393) 
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4h mailing 


Any discussion about the 4th NHES screener mailing, including 
remembering receiving it (or not), the envelope and the contents 
or appearance of the letter. 


Training Example: “I threw this away because I already knew what 
it said.” 


Text Sample: “I picked this [fourth mailing] up for one minute, and I 
wanted to throw it away. It’s pretty bland.” (5707) 


screener design Any discussion about the screener questionnaire design, such as 
the cover, colors, font, layout, etc. 
Training Example: “I wish the pictures were more local.” 
Text Sample: “I really just didn't pay attention to the pictures.” 
(6287) 

screener content Any discussion about the content of the screener questionnaire— 
that is, the instructions or survey questions. 
Training Example: “This question is really confusing.” 
Text Sample: “Yeah, [the screener] guides you over to start over here, 
and it kind of walks you over left to right.” (7471) 

bilingual Any discussion of the bilingual materials including their thoughts 
about the translation or opinions about the Spanish content. 
Training Example: “I read the Spanish first then the English.” 
Text Sample: “I prefer to read it in English many times. Unless Iam 
in a place and they ask me, ‘Do you want English or Spanish?’ And if 
it is well translated, then Spanish; if not [well-translated, then] 
English.” (4711) 

FedEx Any discussion of the FedEx mailer, including remembering 


receiving it (or not). Should be applied with either 2nd screener 
mailing or 4th screener mailing subcode. 


Training Example: “I was surprised to get something from FedEx.” 


Text Sample: “If they're sending it in a FedEx, it must be really 
important.” (6331) 


five-dollar 
incentive 


Any discussion of the five-dollar bill in the initial envelope 
(regardless of whether they remember receiving it). Include 
comments both about the $5 in the example NHES materials and 
the $5 in the actual NHES letter that was sent earlier in the year. 


Training Example: “I wondered who would send me $5.” 
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why not open 


Text Sample: “I don’t know how this one came, but for some reason 
God said open that envelope and there sat a $5 bill.” (4025) 


Discussion of why the participant/household did not or would not 
open a NHES mailing. If participant says they’d open an NHES 
mailing but provide no explanation about why, use the “open” 
subcode within the “adjectives” code. If they provide an 
explanation, double-code using this subcode. 


Training Example: “I don’t open anything that says ‘penalty’ on it.” 


Text Sample: “I was going to [open it], but I was in the process of 
doing some hair, so I literally forgot. ‘Cause I don't like leaving my 
mail on the table when I'm doing hair. I just put it up there, then guess 
I forget that I was supposed to read this.” (7275) 


why open 


Discussion of why the participant/household opened or would 
open a NHES mailing. If participant says they would open an NHES 
mailing but provide no explanation about why, use the “not open” 
subcode within the “adjectives” code. If they provide an 
explanation, double-code using this subcode. 


Training Example: “It looked official, so I opened it.” 


Text Sample: “Whenever I get this type of letter it’s something 
important.” (4655) 


why not complete 


why complete 


Reasons why the participant/household did not complete the 
screener. Use this subcode during any discussion about barriers to 
NHES survey participation, even if the discussion does not directly 
reference the NHES. 


Training Example: “I don’t have time to fill out a survey.” 

Text Sample: “I read it and I see that it doesn’t apply to me.” (5429) 
Reasons why the participant/household did complete the screener. 
Use this subcode during any discussion about facilitators or 


motivations to NHES survey participation, even if the discussion 
does not directly reference the NHES. 


Training Example: “It is important for your voice to be heard.” 
Text Sample: “It’s helping for the future kids.” (6487) 


multiple mailings 


Participants’/household’s reactions to receiving multiple NHES 
mailings. 


Training Example: “I liked receiving several letters because I kept 
misplacing them.” 
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Text Sample: “I do remember getting a second one of these, at which 
time I asked my grandson. I showed him this, and I said to check it out 
to make sure this was really who it says it is. And he said, as near as 
he could find anything out, it was legitimate. So, I was like okay, I'll 
fill it out then.” (4167) 


relevant 


Participant/household believes that the NHES is relevant to them. 
In practice, this may be frequently double coded with “why 
complete.” 


Training Example: “Education affects everyone so it’s important to 
participate.” 


Text Sample: “It’s more about education. And my kids, they'll tell you, 
[I put] more value on education.” (7375) 


not relevant 


Participant/household does not believe that the NHES is relevant 
to them. Include discussion from participants who believe the 
survey is only for people with children or school-aged children. In 
practice, this may be frequently double coded with “why not 
complete.” 


Training Example: “I don’t have kids, so I knew this was not for me.” 


Text Sample: “This has kids all over it. It doesn’t say anything about 
households who have adult children no longer living in them.” (4167) 


recruit NRFU 


materials NRFU 


Any discussion about why the participant or household 
participated in the nonresponse study and/or conversation about 
the recruiting process. 


Training Example: “It is different being asked by a person to do 
something as opposed to a letter.” 


Text Sample: “It’s so much harder to be like ‘No!’ When you’re face- 
to-face than it is to be when something’s coming in the mail that you 
just can’t be bothered with.” (5103) 


Any discussions about the nonresponse study materials, such as 
Sorry We Missed You cards, mailed letters, and CAQ enclosure. 


Training Example: “I liked being able to have a toll-free number to 
call.” 


Text Sample: “I called. And the lady picked up. And I was like, ‘Is this 
a scam?’ She said, ‘No, this is not a scam.’ I said, ‘Okay let's try it.’ 


(5757) 


incentive NRFU 


Any discussion about the $5 or $120 incentives to participate in the 
nonresponse study. 
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Training Example: “Getting $120 made talking to you guys worth 
it.” 

Text Sample: “You want to know why I'm taking this one [survey]? 
For the $120.” (7109) 


how to improve Any discussion or suggestions about how to improve the NHES 
materials. 


Training Example: “The font is so small. If it was bigger it would be 
easier to read without hunting for my glasses.” 


Text Sample: “If there was a deadline on it, like all over, a date, I’m 
sure I would have noticed that.” (4399) 


research Any discussion about how the household members researched or 
investigated the NHES, AIR, NCES, Census, or the follow-up study 
after receiving materials in the mail. 

Training Example: “I googled the 1-800 number but nothing came 


up. y 
Text Sample: “Like I even searched you guys too. I’m like who are 
they?” (6201) 


“Memo” Code 


Definition: When the interviewer memo or field staff notes provide information not 
captured in the transcript, that text will be loaded into NVivo for analysis. The goal of the 
memo subcodes is to capture interviewer and field staff impressions of the interview or 
important interactions and context that would not otherwise be captured in the transcript. 
Only use these subcodes in text that comes from the memos and notes. 


Subcodes 


recruitment Description from the interviewers and field staff about the events 
that led up to recruiting the household to participate in the 
nonresponse study. 


Training Example: “The third time we rang the bell the participant 
answered. She was skeptical and even called the toll-free study 
number. After she spoke to someone at the help desk, she was willing 
to schedule an interview for the following day.” 


Text Sample: “The participant answered the door slightly and stuck 
his head out. We introduced ourselves and the purpose of our visit. He 
suggested that we may want to talk to someone who has kids, but 
after we elaborated on our intent he agreed to the interview for a 
later time.” (4289) 
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neighborhood 


Description from the interviewers and field staff about the 
neighborhood. 


Training Example: “The house is in a tree-lined street with people 
walking their dogs and/or pushing strollers. There were multiple 
stores in the nearby main street.” 


Text Sample: “The lawn was well-manicured and clean, as were most 
of the houses on the block.” (7109) 


home exterior 


home interior 


Descriptions of what the house or property looked like on the 
outside when the interviewer and field staff approached during the 
recruitment or interview phase. 


Training Example: “A broken basketball hoop was tucked behind 
several bags of outgoing household trash.” 


Text Sample: “The front of the house has a metal ramp leading up to 
the front door, and from what we could see the front interior 
staircase had a chairlift to move wheelchairs upstairs.” (5409) 


Descriptions of where the interview took place or what the house 
looked like on the inside. 


Training Example: “The living room was spacious, having two 
recliners and a L-shaped sofa, all in a bright floral pattern.” 


Text Sample: “The house had a lived-in quality to it, slightly 
cluttered, but for the most part organized.” (6081) 


participant conduct 


Interviewer and field staffs’ impressions about the demeanor of or 
interactions with participant and others in the household present 
during the interview. 


Training Example: “The participant laughed a lot and kept being 
distracted by his two-year old grandson who was climbing on the 
chair next to him.” 


Text Sample: “Participant was congenial and hospitable throughout 
the visit, offering us water, turning off the TV.” (4399) 


contradiction 


Interviewers’ and field staffs’ impression that what participants 
said or how they acted were not always in alignment. Use this code 
when interviewer and field staff observe that the participant 
expressed one belief during the interview and expressed a 
contradictory one at another point during the interview. 


Training Example: “At first the participant said he always opened 
all his mail, but we saw piles of unopened mail on the kitchen table 
when we sat down to do the interview.” 
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Text Sample: “His answers were highly inconsistent and often 
contradictory, and we had difficulty adjudicating what was the truth 
and what was something that he thought we wanted to hear.” (6393) 


Adjective” Codes 


Definition: Adjective subcodes are always used in conjunction with a main code or subcode. 
The purpose of adjective subcodes is to add detail, depth, and context about the discussion. 


negative 


Participant’s and/or household's reaction or opinion expresses 
disagreement, or is unfavorable, pessimistic, or adverse. 


Text Sample: “Really, I don't see a benefit. You answer things you 
don’t want to. Some people are honest, others are not and so... I don’t 
know what their real purpose may be. But, for me, I don’t get 
anything from it.” (4711) 


positive 


Participant’s and/or household's reaction or opinion expresses 
agreement or happiness, or is favorable or hopeful. 


Text Sample: “I was a parent that stayed in with the teachers and 
stuff to make sure they were on the right track.” (7481) 


lack 


Participant’s and/or household's reaction or opinion expresses 
apathy or lack of interest. Can also refer to the absence of items or 
materials. 


Text Sample: “To be honest, I don’t even care about the government, 
they do what they’re going to do.” (6081) 


open 


not open 


Participant and/or household opened or would open a mailing. 
Text Sample: “We'll open it, we'll see what it says, if it’s for us, what 
it brings.” (4017) 

Participant and/or household did not open or would not open a 
mailing. 


Text Sample: “This one I probably didn't open.” (7715) 


throw away 


Participant and/or household threw away or would throw away a 
mailing and/or letter. 


Text Sample: “This is going straight to the trash.” (4271) 


keep 


Participant and/or household kept or would keep a mailing and/or 
letter. 


Text Sample: “I just got one of those, so yeah I would keep that.” 
(4373) 
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read Participant and/or household read or would read a mailing and/or 
letter. 


Text Sample: “The first thing I would do—I would look at this, and I 
would make sure that it’s related to me.” (6393) 


would not read Participant and/or household did not read or would not read a 
mailing and/or letter. 


Text Sample: “This is probably a credit card, and I don’t want to be 
bothered with it.” (7109) 


remember Participant and/or household remembers that they received a 
mailing. 


Text Sample: “I remember reading that.” (4167) 


do not remember Participant and/or household does not remember that they 
received a mailing. 


Text Sample: “Well, I don’t remember this. The U.S. Census is 
voluntary.” (5409) 


unclear Not clear from interview what participant and/or household did 
with a piece of mail. 


Text Sample: “You went to the first question, and it was 
over[crosstalk].” (7109) 


I have agency Participant and/or household believes they have influence or 
control over a certain activity, belief, or domain. 


Text Sample: “[Voting is] important. That’s why when my kids are 
18, I make sure they vote.” (7375) 


I do not have Participant and/or household believes they do not have influence 
agency or control over a certain activity, belief, or domain. 


Text Sample: “You know, ain’t nothing going to change. Ain't 
nothing going to happen.” (5515) 


interruption Section of the transcript where the interview is interrupted such as 
the participant taking a phone call or having a conversation with 
another person. 


Text Sample: “Just excuse me for one second, I just want to throw 
something out.” (5463) 
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Please complete the observation protocol below. Record all observations as unobtrusively 
as possible. If necessary, you may explain the purpose of your observations if you are asked 
by any individual in the neighborhood, including anyone from the sampled household. 


DEFINITION 
What do we mean by the sample unit OR the building within which the sample unit 


resides? Here we are making a distinction between: 


e asample unit that we might call a single occupancy dwelling unit (a single 
residential structure that stands alone) and 


e amulti-unit structure where the sample unit resides in a building that houses other 


units. 
GENERAL INFORMATION 
Date of observation: _§_——/___/__ 
Time of day:____:_____ AM / PM (circle one) 


[Take two or three photos of the address you are going to observe. Photos should be taken 
from the front of the address, if possible. Photos should include the entire structure, if possible. 
They should be taken in landscape but can be taken in portrait orientation when needed (e.g., 
for tall buildings). Photos should only be taken from public spaces such as the street, and will 
not be taken while on someone’s private property.] 


OBSERVABILITY 
1. Are you able to observe the sample unit? 


L) Yes > SKIP TO Q3 ON PAGE 2. 

L] No, cannot locate the address 

L] No, can observe the multi-unit building but cannot gain entry to observe the 
unit 

L] No, cannot gain sufficient access to observe the building 


L] No, other reason 


2. Please explain why you are not able to observe the sample unit. 
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IF Q1 = NO, CAN OBSERVE MULTI-UNIT BUILDING BUT CANNOT GAIN ENTRY TO 
OBSERVE UNIT, CONTINUE TO Q3; OTHERWISE, SKIP TO Q8 ON PAGE 6. 


STRUCTURE TYPE 
3. Select the option that best describes the sample unit or the building within which 
the sample unit resides: 


L] Single occupancy dwelling unit (one household in a single residential 
structure that stands alone) 

L] Duplex (two-household structures attached together, but with separate 
entrances) 

L] Townhouse/rowhouse (3 or more household structures attached together, 
but with separate entrances) 

L] Low-rise apartment or condominium building (1-3 floors) 

L] Mid-rise apartment or condominium building (4-6 floors) 

L] High-rise apartment or condominium building (7+ floors) 

L] Commercial business > SKIP TO Q9 ON PAGE 7. 

L) Vacant lot > SKIP TO Q9 ON PAGE 7. 

L] Other non-residential > SKIP TO Q9 ON PAGE 7. 

L] Other - please describe: 


L] Could not determine - please describe: 
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HOUSEHOLD RESIDENTIAL OCCUPANCY STATUS 
4. Based on your observation, indicate whether the sample unit is an/a: 


L] Currently occupied address (e.g., car in the driveway, toys/chairs/yard tools 
outside the house, lights on inside the home) 

L] Seasonal address, no current occupants (e.g., resort area and no evidence of 
current occupants) 

L] Temporarily vacant address (e.g., for sale/rent sign/lock box and no 
evidence of current occupants) 

L] Permanently vacant address (e.g., windows/doors boarded up, condemned 
notice/unlivable conditions, currently under construction) 

L] Other 


L) Could not determine 


Please describe what aspects of the address made you decide on the specific 


occupancy status above. 


IF Q4 = PERMANENTLY VACANT ADDRESS ~ SKIP TO Q8 ON PAGE 6. 


MAIL ACCESS 
5. Based on your observation, how does this sample unit receive mail? 


L] Mail slot 

L] Mailbox attached to their home 

L] Mailbox at the end of a driveway 

L] Mailbox across the street/at the end of the road 
LJ) Mailbox, slot, or room in the multi-unit building 


L] More than one way - please describe: 
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LJ No mailbox or slot in view 


L] Other - please describe: 


L] Could not determine - please describe: 


=> IF Q1=NO, CAN OBSERVE THE MULTI-UNIT BUILDING BUT CANNOT GAIN ENTRY 
TO OBSERVE THE UNIT > SKIP TO Q8 ON PAGE 6. 
=> IF Q4 = TEMPORARILY VACANT ADDRESS, > SKIP TO Q8 ON PAGE 6. 


HOUSEHOLD MEMBER ELIGIBILITY FOR NHES TOPICALS 
6. Based on your observation of the sample unit or building within which the sample 
unit resides, is there any indication that children 18 or under (including babies) may 
live IN THE SAMPLE UNIT (e.g., visible toys, bikes, basketball hoop, sports 
equipment, car seats, strollers, chalk, outdoor swing/play set, bumper sticker or car 
window sticker, child finder sticker for firefighters, etc.)? 


LI Yes 
L] No 


L) Could not determine 


Please describe what aspects of the address made you decide whether or not children 


live in the sample unit. 


HOUSEHOLD ATTRIBUTES 
7. Are there any unique attributes or decorations (e.g., flags, wreaths, bumper stickers, 
signs) on or around the sample unit that indicate -- [f yes, write them down. 
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Pride in education (e.g., honor roll bumper L] Yes 
stickers, university/college flags or decor) 
L] No 
If yes, what? 
Community involvement (e.g., political 
candidate signs, indicators of being involved L] Yes 
with charities, kids’ sports/clubs, neighborhood Oo 
associations) No 
If yes, what? 
Patriotism (e.g., American flag, state flag) L] Yes 
L] No 
If yes, what? 
Privacy (e.g., security camera, no loitering L] Yes 
sign, household alarm sign, locked gate?) AN 
O 


If yes, what? 
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Are there any unique attributes or decorations (e.g., flags, wreaths, bumper 


stickers, signs) on or around the sample unit that indicate -- If yes, write 


them down. 
A language other than English (e.g., signs L] Yes 
or bumper stickers in another language) 
L] No 
If yes, what language? 
If yes, what indicators? 
Anything else 
L] Yes 
L] No 


If yes, what? 


HOUSEHOLD INCOME 
8. Relative to the general population and based on your observations, would you judge 
this sample unit’s household income to be... Use visual cues of the conditions of the 
house/building; number, make, and age of cars at the residence; and knowledge of the 
neighborhood/environment. 


L] In the bottom third of the population 
L] In the middle third of the population 
L] In the top third of the population 


L) Could not determine 
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Please describe what aspects of the address made you decide on the specific 


household income level above. 


9. Provide a general description of the unit and the surrounding neighborhood, 
commenting only on things that have not already been noted in the questions above 
(e.g., building entry requirements, condition of address/neighborhood, objects 
around the address, visible people, other building types in the neighborhood). 


H-8 


